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a reduction  in  training  time  of  approximately  50%  and  to  slightly  higher  scores 
on  criterion-referenced  tests  of  student  knowledge.  There  were  no  substantial 
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between  the  two  student-paced  systems  in  terms  of  either  cost  or  cost  avoidance, 
but  both  were  substantially  less  expensive  than  current  CAI  systems. 
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FOREWORD 


This  research  was  performed  as  part  of  subproject  P13  (Computer-managed 
Instruction)  under  Advanced  Development  Objective  43-03X  (Education  and 
Training  Development).  The  subproject  was  initiated  in  response  to  a 
Technical  Development  Plan  submitted  by  the  Chief  of  Naval  Air  Technical 
Training.  The  work  was  done  under  the  joint  sponsorship  of  the  Chief  of 
Naval  Air  Technical  Training  (later,  the  Chief  of  Naval  Technical  Training) 
and  the  Naval  Personnel  and  Training  Research  Laboratory  (later,  the  Navy 
Personnel  Research  and  Development  Center).  Active  support  and  assistance 
was  provided  at  many  levels  within  the  training  and  research  chains  of 
command. 


The  initial  research  software  package  was  developed  under  contract 
with  Memphis  State  University,  primarily  by  Mr.  B.  Aimes,  Mr.  R.  Dodd, 
and  Mr.  C.  Atkins.  Modifications  of  the  system  were  provided  by  Mr.  C. 
Beckman,  Mr.  G.  Johnson,  and  Ms.  M.  Nix  of  Memphis  State  University, 
and  by  Mr.  R.  Potts,  Mr.  D.  Harvill,  and  DPI  D.  Kaipus  of  the  project 
staff.  Additional  modifications  and  support  in  the  operation  of  the 
system  were  provided  by  Mr.  C.  Tilly  and  the  CMI  group  from  the  Naval  Air 
Station,  Memphis,  Data  Processing  Department. 

The  Naval  Air  Technical  Training  Center,  Memphis,  and,  in  particular, 
the  personnel  of  the  Aviation  Familiarization  Course  and  the  Aviation 
Mechanical  Fundamentals  Course,  deserve  much  of  the  credit  for  the  success 
of  the  project.  Many  school  personnel  participated  activiely  in  the 
derivation  of  course  objectives  and  the  design  of  training  procedures. 

They  provided  the  actual  training  for  all  students  involved  in  the  eval- 
uation of  the  system. 

Most  of  the  course  materials  were  prepared  by  the  following  people: 


Mr.  J.  Andre 
AMSl  L.  Atteberry 
Ms . R.  Berry 
AVCM  M.  Clemons 
Ms.  S.  Ely 
ATC  G.  Erland 
TD2  G . Evans 


Mr.  M.  Evans 
ADJ1  J.  Fielder 
ATI  H.  Fortner 
ADJ2  W.  Knight 
TD1  W.  Mitchell 
ADJ1  D.  Morey 
TDC  D.  Ramey 


TDCS  W.  Saulsberry 
ADJC  D.  Sewell 
Ms.  M.  Smith 
AMSC  M.  Smittle 
Ms.  V.  Weymouth 
PHC  C.  Wright 


All  authors  played  a variety  of  roles  in  the  development  and  evaluation  of 
the  system;  they  have  been  listed  alphabetically. 


JAMES  J.  CLARKIN 
Commanding  Officer 
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SUMMARY 


Problem 


Student-paced  instruction  has  been  shown  repeatedly  to  provide  dramatic 
reductions  in  training  time  with  no  loss  in  student  proficiency.  How- 
ever, the  effective  management  of  large-scale  systems  of  student-paced 
instruction  places  heavy  burdens  on  both  the  classroom  instructor  and 
higher  levels  of  management.  The  purpose  of  this  study  was  to  evaluate 
a system  in  which  the  computer  is  used  to  reduce  some  of  these  burdens. 

Background 


There  have  been  a number  of  reports  on  computer  applications  to  train- 
ing or  education  in  the  last  few  years.  Most  of  them  have  covered  some 
form  of  Computer-assisted  Instruction  (CAI)  in  which  there  is  a one-to- 
one  ratio  between  students  and  terminals  and  in  which  instructional 
materials  are  stored  in  the  computer.  These  studies  have  shown  CAI 
to  be  an  effective  but  rather  expensive  training  technique.  There  have 
also  been  some  studies  of  Computer-managed  Instruction  (CMI)  in  which 
computer  terminals  are  shared  by  a number  of  students  and  in  which 
most  of  the  instructional  materials  are  stored  off  line.  In  general, 
the  CMI  studies  have  been  limited  to  civilian  educational  settings, 
and  interactions  between  the  student  and  the  computer  have  been  infre- 
quent (e.g.,  intervals  of  days  or  even  months). 

Approach 

The  purpose  of  the  present  project  was  to  develop  and  evaluate  a CMI 
system  that  would  be  less  expensive  than  CAI,  would  provide  a frequency 
of  interaction  that  falls  somewhere  between  that  provided  by  CAI  and 
that  normally  provided  by  CMI,  and  would  handle  some  of  the  clerical 
and  administrative  burdens  that  are  normally  imposed  by  student-paced 
instruction.  More  specifically,  a system  was  developed  that  would 
make  assignments,  grade  tests,  provide  feedback  to  the  student,  and 
provide  some  of  the  information  needed  for  the  effective  control  and 
management  of  a large-scale  system  of  student-paced  instruction.  Both 
the  instruction  and  testing  would  take  place  off  line. 

The  system  was  evaluated  in  two  short  courses  taught  at  the  Naval  Air 
Technical  Training  Center,  Memphis.  It  was  compared  to  (1)  conventional 
classroom  Instruction  and  (2)  a system  of  student-paced  instruction 
that  was  based  on  the  training  materials  and  tests  developed  for  the 
CMI  system,  but  which  substituted  ’’manual"  operations  for  certain  of 
the  operations  provided  by  the  computer  in  the  CMI  system.  This  latter 
system  was  called  Instructor-managed  Instruction  (IMI). 
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Results 


It  was  found  that  the  use  of  either  form  of  student-paced  instruction 
(i.e.,  CMI  or  IMI)  provided  a reduction  in  training  time  of  approx- 
imately 50%  and  to  slightly  higher  scores  on  criterion-referenced 
tests  of  student  knowledge.  There  were  no  substantial  differences 
between  the  two  student-paced  systems  in  terms  of  training  effective- 
ness. There  were  several  factors  which  precluded  a precise  comparison 
between  the  two  systems  in  terms  of  either  cost  or  cost  avoidance,  but 
both  were  substantially  less  expensive  than  current  CAI  systems. 

Conclusions 


The  data  from  this  study  suggest  that  both  the  CMI  system  and  IMI  system 
are  cost-effective  training  techniques.  This  particular  CMI  system 
contained  a number  of  "manual"  elements  that  could  be  eliminated.  The 
IMI  system,  on  the  other  hand,  was  dependent  on  the  computer  in  several 
respects,  and  the  elimination  of  this  support  might  have  seriously 
degraded  the  system's  efficiency.  Because  of  these  overlapping  areas, 
among  other  reasons,  a choice  between  the  two  systems  requires  a con- 
sideration of  factors  beyond  those  reflected  directly  in  data  gathered 
during  the  evaluation.  On  the  basis  of  such  considerations,  the  Naval 
Technical  Training  Command  has  chosen  to  proceed  with  the  higher  level 
of  computer  support  (CMI). 

There  are  many  features  of  the  CMI  system  used  during  the  evaluation 
that  were  based  on  arbitrary  decisions  rather  than  empirical  evaluations 
of  alternatives.  It  is  probable  that  refinements,  based  on  empirical 
evaluations,  could  substantially  improve  the  system  efficiency. 
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AN  EVALUATION  OF  COMPUTER- MANAGED  INSTRUCTION 
IN  NAVY  TECHNICAL  TRAINING 

BACKGROUND 

During  the  mid-1960s,  considerable  work  was  initiated  in  the  area  of 
adaptive  instruction  mediated  by  computers.  A major  part  of  this  work 
used  a tutorial  mode  in  which  instruction  was  carried  on  by  means  of  a 
dialogue  between  the  computer  and  the  student.  This  type  of  instruction 
usually  was  referred  to  as  CAI  (Computer  Assisted  Instruction,  or  Com- 
puter Aided  Instruction).  Other  applications  of  the  computer  in  the  in- 
structional process  had  been  conceived,  but,  on  the  whole,  had  not  been 
developed  to  an  appreciable  extent.  One  of  these  computer  applications 
has  been  termed  Computer  Managed  Instruction  (CMI).  CMI  sought  to  retain 
as  many  of  the  advantages  of  tutorial  CAI  as  possible,  while  holding  the 
cost  of  the  instructional  system  to  a minimum.  The  economic  advantages 
of  such  a system  were  to  be  accomplished  primarily  through  increased  use 
of  off-line  instruction  and  a reduction  in  the  number  of  interactions 
between  the  computer  and  the  student. 

In  fiscal  year  1968  the  project  which  was  to  become  the  Navy's  pri- 
mary undertaking  in  the  area  of  CMI  was  initiated  jointly  by  the  Chief 
of  Naval  Air  Technical  Training  and  the  Navy  Training  Research  Laboratory 
Branch  Office,  Memphis.  Certain  historical  aspects  of  the  project,  in- 
cluding funding  support  from  several  sources,  are  described  briefly  in 
Appendix  A.  The  objective  of  the  project  was  to  test  the  feasibility  of 
the  CMI  concept  through  the  development  and  evaluation  of  a CMI  system 
suitable  for  use  in  a large-scale  technical- training  environment.  The 
present  report  is  a description  of  this  effort. 

GENERAL  ORIENTATION 

The  term  CMI  has  been  applied  to  a wide  diversity  of  systems.  Many 
of  these  systems  have  focused  on  the  support  of  rather  high-level  adminis- 
trative functions  while  providing  little  or  no  immediate  support  for  the 
management  of  the  instructional  process  itself.  The  CMI  system  developed 
under  the  current  project  differs  from  those  applied  to  administrative 
functions  in  that  it  focuses  on  the  direct  management  of  the  student's 
interactions  with  the  instructional  materials  and  on  the  support  of  certain 
functions  that  are  intimately  associated  with  these  interactions.  It 
provides  relatively  little  In  the  way  of  support  at  the  higher  levels  of 
management.  This  emphasis  reflects  the  allocation  of  limited  resources 
to  what  is  felt  to  be  the  more  interesting  and  controversial  class  of 
problems.  It  does  not  imply  that  computer  support  of  certain  purely  ad- 
ministrative functions  is  unimportant  or  that  there  is  any  basic  incompat- 
ability  between  the  two  kinds  of  support. 
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The  emphasis  described  above  is  obviously  quite  similar  to  the  emphasis 
found  in  most  CAI  systems.  As  mentioned  previously,  the  initial  approach 
to  this  particular  CM1  system  grew  out  of  a consideration  of  the  ways  in 
which  the  cost  of  tutorial  CAI  could  be  reduced  without  seriously  degrading 
effectiveness.  A brief  review  of  these  considerations  may  provide  a useful 
introduction  to  the  system  that  was  actually  developed  under  this  project. 

One  of  the  first  major  changes  to  be  considered  was  the  presentation 
of  the  actual  instructional  material  off  line.  This  would  radically  reduce 
the  amount  of  material  that  must  be  stored,  retrieved,  and  displayed  by 
the  computer.  Without  additional  changes  in  other  aspects  of  the  system, 
however,  a simple  change  of  this  kind  would  necessitate  a rather  complicated 
and  cumbersome  system  for  the  location  and  retrieval  of  these  off-line 
materials . 

Hie  second  major  change  to  be  considered  was  a reduction  in  the  number 
of  interactions  between  the  student  and  the  computer.  Most  interactions 
in  the  tutorial  mode  are  devoted  to  testing  the  student  and,  on  the  basis 
of  these  tests,  providing  him  with  either  new  or  remedial  instructional 
material.  Although  certain  CAI  systems  provide  fairly  sophisticated  ways 
to  handle  these  interactions,  there  have  been  many  programs  written  in 
which  most  or  all  interactions  are  handled  by  techniques  that  are  func- 
tionally identical  to  those  available  in  standard  programmed  instruction. 

It  was  felt  that  if  interactions  of  this  kind  were  handled  off  line,  the 
major  effect  would  be  on  the  strictness  of  control  (i.e.,  does  the  student 
really  do  what  he  Is  supposed  to  do) , rather  than  on  the  actual  form  of 
the  interaction.  Studies  with  teaching  machines  have  shown  that  there 
are  many  instructional  situations  in  which  strictness  of  control  is  not 
particularly  important. 

If  the  student-program  interactions  for  moderately  large  blocks  of 
instructional  material  could  be  handled  off  line,  the  problems  of  locating 
this  material  and  of  shifting  back  and  forth  between  it  and  a computer 
terminal  would  be  greatly  reduced.  The  primary  advantage,  however,  would 
be  a system  in  which  a number  of  students  share  a single  terminal. 

The  remaining  considerations  were  directed  toward  changes  that  would 
have  lesser  effects  on  either  costs  or  effectiveness.  Within  a system 
of  the  type  discussed  in  this  report,  for  example,  most  of  the  interactions 
between  the  student  and  the  computer  would  take  the  form  of  dispersed  blocks 
of  test  materials  that  would  provide  both  strict  control  of  performance 
and  the  basis  for  diagnosis,  assignment,  and  tracking  by  the  computer. 

These  tests  could  be  taken  on  line  at  the  terminal  or  they  could  be  taken 
off  line,  with  student  responses  recorded  in  machine-readable  form.  The 
latter  alternative  was  selected  for  the  system  used  during  the  evaluation. 
Ihis  placed  some  limitations  on  the  type  of  questions  and  responses  that 
could  be  handled  gracefully,  but  it  also  kept  the  student  from  monopolizing 
the  terminal  during  the  time  when  he  was  reading  questions,  thinking  about 
his  answers,  and  entering  his  responses.  This,  in  turn,  increased  the 
number  of  students  that  could  share  a single  terminal. 
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In  summary,  an  instructional  system  was  designed  in  which  most  of 
the  instructional  materials  would  take  the  form  of  off-line,  self-admin- 
istered packages  or  modules,  and  most  of  the  testing  would  be  done  off 
line,  with  answers  recorded  on  machine-readable  answer  sheets.  The  com- 
puter would  be  used  for  such  things  as  test  scoring,  diagnosis  of  student 
deficiencies,  assignment  of  tests,  assignment  of  new  and  remedial  instruc- 
tional materials,  tracking  of  students  through  the  materials,  and  prep- 
aration of  various  reports  required  for  student  management.  Computer 
terminals  would  be  shared  by  a number  of  students. 

The  following  section  describes  the  system  used  during  the  evaluation. 
Minor  changes  have  been  made  since  the  evaluation,  and  there  will  be  others 
in  the  future. 


SYSTEM  DESCRIPTION 


Assignment  Patterns 


A CMI  course  consisted  of  a collection  of  instructional  modules. 

A student  could  register  for  the  course  under  one  of  several  patterns. 

The  pattern  provided  a means  for  controlling  the  assignment  of  modules. 

It  determined  which  modules  out  of  the  total  set  would  actually  be  assigned, 
the  general  sequence  in  which  the  modules  would  be  assigned,  and  which 
modules  had  to  be  completed  successfully  before  certain  others  could  be 
assigned. 

Learning  Guides 


The  computer  communicated  with  the  student  by  means  of  Learning 
Guides.  These  guides  had  several  general  elements,  including  the  follow- 
ing: 


1.  Heading 

The  heading  contained  information  such  as  date,  time,  course  identi- 
fication, and  the  student fs  name,  social  security  number,  learning  center, 
and  carrel  number. 

2.  New  Assignments 

This  element  referred  the  student  to  a learning  module  and  a test 
to  be  taken  at  the  completion  of  the  module.  It  indicated  the  volume 
and  page  where  the  test  could  be  found,  the  test  identification  number 
that  the  student  was  to  put  on  his  answer  sheet,  and,  when  a latent  image 
answer  sheet  was  to  be  used,  the  form  of  the  answer  sheet  and  the  letter 
that  would  indicate  a correct  response.  It  also  contained  any  special 
instructions  that  might  be  required  for  the  module  in  question.  Although 
the  module  test  volumes  were  available  to  the  student  when  needed,  the 
end-of-course  exams  had  to  be  obtained  from  the  instructor.  Trainees  were 
monitored,  and  were  required  to  remove  training  materials  from  the  carrels 
while  taking  tests. 
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3.  Remedial  Assignments 


This  element  contained  much  the  same  information  as  the  one  for 
new  assignments.  However , in  most  cases,  the  student  was  told  the  part 
or  parts  of  the  module  on  which  he  had  failed  to  meet  a criterion,  and 
was  assigned  tests  that  cover  only  these  parts.  When  the  student  had 
taken  a specified  number  of  remedial  tests  in  a given  area  without  meeting 
the  criterion,  he  was  told  to  report  to  the  instructor  for  more  assistance 
in  that  particular  area.  The  tests  used  for  both  new  and  remedial  assign- 
ments were  randomly  selected  without  replacement  from  among  sets  of  par- 
allel tests. 

4.  General  Feedback 

The  student  was  informed  when  he  had  completed  a module  and  when 
he  had  completed  certain  blocks  of  modules.  The  system  has  a capability 
for  providing  question-by-question  feedback,  but  this  capability  was  not 
used  during  the  period  of  the  evaluation. 

Answer  Sheets 


Students  communicated  with  the  computer  by  means  of  optically- readable 
answer  sheets.  In  the  heading  of  these  sheets  were  blocks  for  the  written 
and  coded  identification  of  both  student  and  test.  At  the  time  of  the 
evaluation  only  two  types  of  answer  sheets  were  being  used.  The  first 
was  a simple  five  alternative,  50-question,  multiple-choice  answer  sheet 
that  was  used  for  end-of-course  tests  and  special  inputs  (e.g. , following 
shop  work).  The  second  answer  sheet  was  similar  to  the  first,  but  was 
printed  with  a special  ink  so  that  when  an  answer  alternative  was  rubbed 
with  a special  crayon,  one  of  five  letters  appeared.  Since  letters  could 
not  be  scanned  by  the  equipment  being  used  during  the  evaluation,  when 
the  student  made  an  error  he  was  required  to  make  a pencil  mark  in  an  error 
box  at  the  beginning  of  the  answer  line.  There  were  15  basic  patterns 
for  these  latent  image  answer  sheets,  and  since  any  one  of  the  five  letters 
could  be  designated  as  correct,  there  were  75  answer  patterns  available. 

The  cost  was  $0,016  per  sheet,  in  quantity. 

The  system  has  capabilities  for  accepting  other  than  multiple- choice 
responses,  but  these  capabilities  were  not  being  used  during  the  eval- 
uation. When  constructed  responses  were  required  as,  for  example,  in 
the  preparation  of  a Maintenance  Action  Form,  the  student  was  instructed 
to  recode  his  responses  into  a multiple-choice  format. 

Records  and  Reports 

The  computer  maintained  records,  for  each  student,  of  when  modules 
were  assigned,  when  tests  were  submitted,  which  questions  were  missed, 
and  when  modules  were  completed. 
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Information  was  provided,  either  for  individual  students  or  for  all 
students,  on  current  assignments  and  the  time  at  which  each  had  been  made. 
Data  on  student  aptitude  (Basic  Test  Battery  Scores)  were  used  to  form 
multiple  regression  equations  that  could  be  used  to  predict  the  student's 
position  within  the  course  after  various  periods  of  time  within  the  course. 
The  difference  between  actual  position  within  the  course  and  predicted 
position  within  the  course  was  used  as  the  basis  for  a daily  roster  of 
(1)  students  who  should  be  sent  to  night  school,  and  (2)  students  who 
were  in  danger  of  being  sent  to  night  school  if  their  performance  did 
not  improve  (Johnson,  Salop,  & Harding,  1972).  A daily  roster  was  also 
provided  that  listed  predicted  time  to  course  completion  for  each  student 
in  the  course.  Finally,  each  student  who  completed  a course  was  assigned 
a grade  that  represented  a weighted  average  of  his  performance  on  the 
end-of-course  examination , time  in  the  course,  and,  when  appropriate, 
performance  in  the  shop. 

Hardware 


During  the  period  of  the  evaluation,  the  system  was  run  on  a Xerox 
Data  Systems  Sigma  9 computer  located  at  Memphis  State  University.  This 
computer  had  256,000  characters  of  internal  storage  and  four  disk  pack 
drives  providing  an  additional  100  million  characters  of  external  storage. 
This  computer  was  dedicated  to  Navy  use  during  regular  training  hours, 
and  was  backed  up  by  a second  Sigma  9 computer  that  was  normally  used 
by  the  university  itself. 

Four  remote  batch  terminal  clusters  were  used  during  the  evaluation. 

One  supported  the  two  CMI  learning  centers,  one  supported  five  "non-CMl" 
learning  centers  in  which  the  same  material  was  being  taught  (including 
two  that  were  used  in  the  evaluation) , one  supported  a CMI  learning  center 
In  the  Basic  Electricity  and  Electronics  portion  of  the  Avionics  Technician 
course,  and  one  was  used  for  developmental  work  and  troubleshooting. 

Each  of  these  clusters  consisted  of  a Data  100  Corporation  terminal,  Model 
78,  supplemented  by  an  Optical  Scanning  Corporation  optical  mark  reader, 
Model  100DC,  and  card  punch,  Model  521.  This  terminal  configuration  could 
accept  approximately  40  answer  sheets  per  minute  and  could  produce  learning 
guides  at  the  rate  of  about  300  lines  per  minute.  It  was  estimated  that 
each  terminal  could  support  up  to  200  students. 

PROCEDURES 


Courses 


The  evaluation  was  done  in  two  relatively  short  courses,  the  Aviation 
Familiarization  Course,  Class  P (AFAM) , and  the  Aviation  Mechanical  Fund- 
amentals Course,  Class  P (AMFU).  The  AFAM  course  provided  training  on 
a number  of  general  topics  related  to  aircraft,  aircraft  handling,  and 
aircraft  carriers.  It  also  provided  separate  training  for  Navy  and  Marine 
Corps  students  on  aviation  ratings  or  MDSs  and  the  organizational  struc- 
ture of  aviation  units  in  the  two  services  (for  a complete  outline,  see 
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Appendix  B).  The  AMFU  course  provided  training  on  topics  that  are  common 
to  a variety  of  maintenance  Jobs  in  the  aviation  community— 'for  example, 
the  use  of  common  aviation  hand  tools,  corrosion  control,  and  various 
elements  of  the  Naval  Aviation  Maintenance  Program  (for  a complete  outline 
see  Appendix  B).  In  their  original  forms,  the  AFAM  course  was  2 weeks 
and  the  AMFU  course,  4 weeks. 

Specific  training  objectives  were  not  available  for  the  original  AFAM 
and  AMFU  courses.  Project  personnel  worked  with  school  personnel  in  the 
derivation  of  the  objectives  that  would  be  needed  for  the  development 
of  instructional  materials  for  the  CMI  system.  The  objectives  were  sub- 
sequently adopted  by  the  schools,  and  new  courses  were  developed  that 
were  oriented  specifically  toward  the  teaching  of  these  objectives.  The 
new  courses  were  intiated  several  months  prior  to  the  beginning  of  the 
evaluation,  providing  ample  time  for  revision  and  for  the  instructors 
to  become  familiar  with  the  new  lessons.  The  revised  AFAM  course  was 
1 week,  and  the  revised  AMFU  course,  2 weeks. 

Individualized  Instructional  Materials 


The  courses  were  divided  into  modules,  each  covering  a set  of  related 
training  objectives.  Average  training  time  on  thes  modules  varied  from 
approximately  15  to  120  minutes.  There  were  12  modules  in  the  AFAM  course 
and  20  in  the  AMFU  course. 

All  modules,  except  certain  shop  modules  that  were  taught  by  means  of 
slide-tape  programs,  were  taught  by  means  of  programmed  instruction  booklets. 
A variety  of  programming  techniques  were  used.  To  capitalize  on  the 
system's  capability  for  remediation,  a concerted  effort  was  made  to  avoid 
overprograraming.  However,  the  need  for  remediation  differed  considerably 
from  module  to  module.  Data  from  a sample  of  100  students  who  took  both 
courses  indicated  that  in  the  AFAM  course  there  was  an  average  of  .25 
remedial  assignments  per  student  per  module  (with  a range  over  modules 
of  .00  to  .97),  and  an  average  of  2.65  remedial  assignments  on  the  final 
comprehensive  examination.  The  AMFU  course  averaged  .65  remedial  assign- 
ments per  student  per  module  (with  a range  over  modules  of  .01  to  1.98) 
and  an  average  of  12.34  remedial  assignments  on  the  final  comprehensive 
examination. 

Each  program  contained  a self-test  covering  all  objectives.  If  a 
student  felt  that  he  might  already  know  the  material  taught  in  the  program, 
he  could  look  at  this  self- test  and  determine  whether  or  not  he  should 
read  the  program.  The  self-tests  were  not  graded,  but  the  student  was 
told  that  he  should  be  certain  of  his  ability  to  pass  these  tests  before 
he  took  the  tests  that  would  be  submitted  to  the  computer. 

Most  of  the  instructional  materials  were  developed  by  Navy  enlisted 
men  who  had  attended  a 13-day  instructional  programmers  course  taught  at 
NATTC  Memphis.  They  worked  under  the  direct  supervision  of  a civilian 
training  specialist. 
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Training  Methods 


Several  features  of  the  CMI  system  used  during  the  evaluation  were 
the  result  of  constraints  that  had  been  encountered  during  the  system's 
development  or  of  fairly  rapid  adaptations  made  in  response  to  deficiencies 
discovered  during  the  Initial  large-scale  implementation  of  the  system 
in  the  schools.  The  reasons  for  certain  procedures  used  during  this  period 
can  be  understood  only  within  this  historical  context. 

The  system  operated  in  the  remote  batch  mode.  This  type  of  operation 
was  originally  determined  by  hardware  limitations  that  existed  early  in 
the  development.  These  hardware  limitations  affected  the  software,  and 
this,  in  turn,  influenced  the  selection  of  new  hardware,  even  after  the 
original  limitations  had  ceased  to  exist.  Because  of  the  delays  imposed 
by  the  batch  operation  itself  and  the  time  spent  in  transporting  answer 
sheets  and  learning  guides  between  the  individual  learning  centers  and 
the  central  terminal  site,  there  was  a considerable  wait  between  the 
student's  submission  of  an  answer  sheet  and  his  receipt  of  a response 
from  the  computer.  To  minimize  lost  time,  the  student  was  provided  alter- 
native modules  on  which  he  could  work  while  waiting  for  the  computer's 
response.  He  was  given  an  initial  assignment  of  several  modules  and  was 
assigned  a new  module  whenever  he  completed  an  old  one. 

The  original  implementation  in  the  schools  indicated  that  these  pro- 
cedures were  fairly  effective  in  keeping  the  student  engaged  in  construc- 
tive learning  activities,  even  though  there  were  still  cases,  particularly 
if  he  received  a number  of  remedial  assignments  near  the  end  of  a course, 
when  he  would  simply  have  to  wait.  However,  there  were  other  problems 
that  arose  from  the  multiple  assignments  themselves.  If  something  went 
wrong,  if  the  student  forgot  or  misplaced  an  assignment,  if  he  miscoded 
an  answer  sheet,  if  answer  sheets  stuck  together  while  going  through  the 
scanner,  or  if  the  computer  simply  failed  to  evaluate  an  answer  sheet, 
the  failure  would  often  remain  undiscovered  until  long  after  it  had  occur- 
red. In  such  cases  the  student  would  eventually  find  himself  with  nothing 
to  do  while  awaiting  further  assignments  that  were  actually  blocked  by 
a missing  prerequisite.  When  it  was  finally  discovered  that  something 
must  have  gone  wrong,  the  student  and  instructor  might  spend  as  much  as 
several  hours  trying  to  locate  and  correct  the  deficiency. 

Block  assignments,  the  manual  checking  of  both  answer  sheets  and 
learning  guides,  and  the  limited  use  of  instructor-assigned  remediation 
were  all  introduced  as  short  term  solutions  to  the  problems  described 
above. 

Computer-Managed  Instruction 

The  CMI  instruction  was  given  in  two  learning  centers,  an  80-student 
center  and  a 49-student  center.  Instruction  took  place  in  individual 
carrels  or  in  a shop  area  adjacent  to  the  center.  Students  remained  in 
the  same  carrels  through  both  courses.  The  two  learning  centers  were 
serviced  by  a single  terminal  located  in  the  same  building. 
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As  an  aid  in  checking  answer  sheets  against  learning  guides,  the 
instructional  modules  were  generally  grouped  into  blocks  of  from  three 
to  five  modules  each.  The  block  would  be  assigned  as  a unit  and  the 
student  would  retain  his  answer  sheets  until  he  had  completed  all  modules 
in  the  block.  He  would  then  take  both  the  learning  guide  and  answer  sheets 
to  the  instructor  who  would  check  the  answer  sheets  for  coding  errors 
and  make  sure  that  a sheet  had  been  submitted  for  each  assignment. 

The  instructor  maintained  a folder  on  each  student  in  which  was  kept 
previously  generated  learning  guides  that  covered  new  blocks  of  material 
and  pending  remedial  assignments.  When  the  student  turned  in  one  assign- 
ment, he  was  given  a new  learning  guide.  The  old  learning  guide  was 
retained  in  the  folder  until  a response  was  received  from  the  computer. 

A runner  would  periodically  pick  up  student  answer  sheets  from  the 
learning  center  and  deliver  them  to  the  terminal  room.  He  would  also 
deliver  learning  guides  from  the  terminal  room  to  the  learning  center. 

When  a learning  guide  was  brought  to  the  learning  center,  the  instructor 
would  check  it  against  the  old  learning  guide  in  the  student’s  folder 
to  make  sure  that  a response  had  been  received  for  each  of  the  answer 
sheets  that  had  been  submitted.  When  the  computer  indicated  that  the 
student  had  successfully  completed  all  modules  in  a block,  the  original 
learning  guide  for  that  block  was  discarded  from  the  folder. 

To  avoid  delays  resulting  from  an  extended  series  of  remedial  assign- 
ments, the  instructor  himself  would  make  the  assignment  when  a student 
required  more  than  the  initial  remedial  assignment  provided  by  the  com- 
puter. The  instructor  was  given  a guide  that  contained  grading  standards 
and  the  detailed  information  needed  to  assign  both  tests  and  answer  sheets. 

Instructor-managed  Instruction 

It  was  originally  intended  to  have  a control  group  which  would  be 
taught  by  individually-paced  instruction,  but  in  which  all  or  most  of 
the  functions  provided  by  the  computer  under  CMI  would  be  provided  by 
the  instructors.  To  avoid  imposing  serious  administrative  burdens  on 
the  school  personnel,  hcwever,  a moderate  level  of  computer  support  was 
eventually  provided.  This  treatment  will  be  referred  to  as  Instructor- 
managed  Instruction  or  IMI. 

IMI  instruction  was  provided  in  two  learning  centers  that  were  similar 
to  those  used  for  CMI,  an  80-student  and  a 57-student  center.  All  in- 
structional materials  and  tests  were  exactly  the  same  as  those  used  for 
CMI. 


Most  features  of  the  IMI  management  system  were  quite  similar  to  those 
of  the  CMI  system.  The  main  differences  were  that  all  tests  except  the 
end-of-course  examination  were  evaluated  by  the  instructor,  and  all  assign- 
ments except  the  initial  remedial  assignments  following  the  end-of-course 
examination  were  made  by  the  instructor.  Modules  were  grouped  into  blocks 
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and  students  submitted  answer  sheets  for  the  complete  block,  just  as  they 
did  in  the  CMI  system.  Individual  folders  were  maintained  on  each  student. 
Initial  assignments  were  made  by  means  of  a set  of  pre-printed  learning 
guides  that  were  kept  in  each  student's  folder.  The  instructors  evaluated 
the  answer  sheets  and  made  remedial  assignments  on  the  basis  of  information 
provided  in  guides  of  the  same  general  type  as  those  used  by  the  CMI 
instructors  in  making  repeated  remedial  assignments. 

Each  of  the  IMI  students  was  actually  registered  in  a modified  CMI 
system  that  provided  rosters  of  the  students  in  each  learning  center, 
scored  the  end-of-course  examinations  and  provided  the  initial  remedial 
assignments  on  these  examinations,  kept  track  of  the  amount  of  time  that 
the  student  spent  in  the  course,  and  provided  a final  grade  for  the  course. 
Ihese  two  centers,  plus  three  additional  IMI  learning  centers  which  were 
not  Included  in  the  evaluation,  were  served  by  a single  terminal. 

Conventional  Instruction 


Conventional  Instruction  (Cl)  was  provided  in  six  classrooms,  each 
containing  a section  of  from  20  to  25  students.  Two  new  sections,  one 
of  Navy  students  and  one  of  Marine  students,  started  each  Monday.  Instruc- 
tion was  by  means  of  lectures  and  discussions,  supplemented  by  films, 
charts,  and  transparencies.  Each  student  was  provided  with  a workbook 
that  contained  a complete  list  of  the  objectives  taught  in  each  lesson, 
together  with  sample  test  questions  of  the  kind  provided  on  the  self- 
tests of  the  CMI  and  IMI  program.  The  student  could  answer  the  questions 
during  the  class  or  in  the  barracks  at  night.  The  preceding  day's  answers 
were  checked  by  the  instructor  each  morning.  A student  who  did  poorly 
on  the  end-of-course  examination  could  be  sent  back  to  repeat  all,  or, 
in  the  case  of  AMFU,  part  of  the  course. 

Subjects 

The  subjects  used  in  the  evaluation  were  selected  from  the  population 
of  students  who  entered  the  AFAM/AMFU  sequence  between  30  April  and 
14  June  1973,  and  who  were  scheduled  for  subsequent  assignment  to  the 
Basic  Helicopter  Course,  Class  C,  or  to  one  of  the  following  Class  A 
courses:  Avionics  Technician,  Aviation  Structural  Mechanic  S (Structures), 

Aviation  Structural  Mechanic  H (Hydraulics),  Aviation  Structural  Mechanic 
E (Support  Equipment),  Aviation  Machinist's  Mate  J (Jet  Engine),  Aviation 
Machinist's  Mate  R (Reciprocating),  Aviation  Support  Equipment  Technician 
Electrical,  Aviation  Support  Equipment  Technlcial  Hydraulics  and  Structures, 
or  Aviation  Support  Equipment  Technician  Mechanical.  Students  needed 
to  make  up  conventional  sections  and  to  fill  vacancies  in  the  CM!  or  IMI 
learning  centers  were  randomly  selected  from  the  pool  of  students  available 
at  any  given  time;  the  remaining  students  were  assigned  to  learning  centers 
that  were  not  used  in  the  evaluation.  These  procedures  led  to  variations 
in  the  number  of  students  trained  under  the  different  methods.  A minimum 
of  142  Navy  students  and  99  Marine  students  completed  training  under  each 
method. 
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Criteria 


The  following  criteria  were  used  in  the  study: 

1.  Drops 

Students  were  separated  into  three  categories:  (1)  those  who 

successfully  completed  both  the  AFAM  and  AMFU  courses,  (2)  those  who  were 
dropped  from  one  or  the  other  of  the  two  courses  because  of  difficulties 
with  the  material  (lack  of  knowledge,  interest,  or  aptitude),  and  (3) 
those  who  were  dropped  from  one  or  the  other  of  the  two  courses  for  reasons 
unrelated  to  the  courses  themselves  (discipline,  medical,  etc.).  Students 
in  the  latter  category  were  eliminated  from  further  consideration.  This 
variable  is  simply  the  percentage  of  students  dropped  because  of  diffi- 
culties in  learning  the  material. 

2.  Time  to  Completion 

In  all  versions  of  the  courses,  this  variable  consisted  of  normal 
classroom  time  (including  time  during  setbacks  in  the  conventional  sec- 
tions), plus  time  spent  in  night  school. 

3.  Percentage  Correct 


For  both  courses,  the  percentage  correct  consisted  of  the  per- 
centage of  items  that  the  student  got  right  the  first  time  he  attempted 
the  end-of-course  examination.  These  examinations  provided  an  exhaustive 
coverage  of  all  objectives  except  the  performance  objectives  taught  in 
the  shop  portions  of  the  AMFU  course.  The  AFAM  test  for  Navy  students 
contained  220  items;  and  for  Marine  students,  182  items.  The  AMFU  test 
contained  336  items  for  both  Navy  and  Marine  students.  The  CMI  and  IMI 
students  were  given  remedial  assignments  on  any  areas  in  which  they  failed 
to  meet  criteria.  The  improvement  that  resulted  from  those  remedial 
assignments  is  not  reflected  in  the  Percentage  Correct  scores,  but  the 
time  required  was  included  in  the  Time  to  Completion  scores.  The  improve- 
ment resulting  from  setbacks  in  the  conventional  versions  of  the  courses 
was  included  in  the  scores. 

4.  Shop  Grade 

The  AMFU  course  contains  several  shop  exercises,  each  of  which 
is  scored  by  the  instructor  in  a detailed  checklist.  Errors  are  weighted 
in  accordance  with  standardized  estimates  of  importance.  This  variable 
was  a linear  transformation  of  the  total  score  on  these  checklists. 
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5.  Attitude  I 


Upon  his  completion  of  the  AMFU  course,  each  student  was  given 
an  11- item  questionnaire  on  his  attitude  toward  both  courses.  Each  item 
was  in  a 5-point  Likert  format,  with  0 representing  a very  unfavorable 
response,  2 representing  a neutral  response,  and  A representing  a very 
favorable  response.  This  variable  was  the  simple  average  of  the  11  items. 
Questions  used  are  given  in  Appendix  C. 

6.  Attitude  II 

Each  student  in  the  CMI  and  IMI  sections  was  also  given  an  8-item 
questionnaire  on  his  attitude  toward  the  instruction  he  had  received 
relative  to  his  attitude  toward  conventional  instruction.  Question  format 
and  scoring  was  similar  to  that  used  for  Attitude  I.  Questions  used  are 
given  in  Appendix  C. 

7.  Delayed  Retention 

All  items  on  the  AFAM  and  AMFU  end-of-course  tests  were  split 
between  two  complementary  delayed  retention  tests.  Each  of  the  students 
from  the  evaluation  who  went  into  the  Basic  Helicopter  Course  or  into 
one  of  the  Aviation  Machinist’s  Mate  or  Aviation  Structural  Mechanic  courses 
was  given  one  or  the  other  of  these  delayed  retention  tests  approximately 
6 weeks  after  his  graduation  from  the  AMFU  course.  This  variable  consists 
of  the  percentage  of  items  that  the  student  answered  correctly. 

8.  Class  A School  Grade 


This  variable  was  simply  the  final  average  in  the  subsequent  Class 
A school  for  students  from  the  evaluation  groups  who  went  into  the  Basic 
Helicopter  Course  or  into  one  of  the  Aviation  Machinist’s  Mate  or  Aviation 
Structural  Mechanic  courses.  It  was  used  as  a criterion  because  some 
of  the  material  taught  in  the  AMFU  course  is  viewed  as  prerequisite  mater- 
ial for  these  courses. 


RESULTS 


Most  analyses  were  done  by  ANOVAs  using  unweighted  means.  In  these 
cases,  the  means  of  the  cell  means  are  reported  rather  than  the  means 
of  the  raw  data.  A number  of  the  analyses  used  week-of-entry  as  one  of 
the  independent  variables.  This  was  done  primarily  as  a guard  against 
fluctuations  in  the  quality  of  student  input  or  unexpected  administrative 
changes.  Most  analyses  also  used  student  aptitude  as  an  independent  vari- 
able. For  Navy  students,  this  variable  consisted  of  the  simple  sum  of 
scores  on  the  General  Classification  Test,  the  Arithmetic  Test,  the 
Mechanical  Test,  and  the  Clerical  Test.  For  Marine  students,  This  vari- 
able consisted  of  the  simple  sum  of  scores  on  Verbal  Expression  Test, 
the  Arithmetic  Reasoning  Test,  the  Mechanical  Aptitude  Test,  and  the 
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Pattern  Analysis  Test,  Previous  research  has  indicated  that  these  com- 
posites correlate  with  the  various  dependent  variables  used  in  the  eval- 
uation of  a level  that  approaches  those  provided  by  multiple  correlations. 

In  the  case  of  Navy  students,  all  students  were  divided  into  three  groups 
of  approximately  equal  sizes  by  counting  down  on  this  variable.  A similar 
procedure  was  used  in  the  case  of  certain  analyses  on  the  Marine  students. 
However,  for  most  analyses  the  Marines  were  divided  into  only  two  groups 
to  avoid  cells  containing  less  than  two  students. 

Differences  discussed  in  this  section  are  reliable  at  the  .01  level. 
Differences  between  levels  of  aptitude,  unless  specifically  noted,  are 
all  reliable  at  this  level.  Contrasts  between  individual  treatments  were 
made  by  means  of  what  Weiner  (1962)  refers  to  as  the  Tukey  (a)  test. 

The  actual  tables  for  the  ANOVAs  are  in  Appendix  D.  Tables  of  cell  means 
are  in  Appendix  E. 

Tables  1 and  2 provide  separate  summaries  of  the  results  for  Navy 
and  Marine  students.  However,  in  most  cases  the  patterns  are  quite  similar, 
so  both  can  be  discussed  at  the  same  time. 

Drops 

The  differences  in  drops  were  tested  by  means  of  X2  and  were  found 
to  be  unreliable.  However,  these  figures  may  not  provide  an  unbiased 
or  objective  index  of  training  effectiveness.  The  standards  used  to 
evaluate  marginal  students  in  the  CMI  and  IMI  centers  were  based  in  part 
on  training  time,  and  differed  from  those  used  in  classroom  instruction 
(Cl).  Although  each  of  these  standards  was  objective,  it  is  quite  pos- 
sible that  they  were  not  equally  stringent.  For  a student  to  be  dropped 
for  academic  reasons,  he  first  had  to  be  identified  as  performing  in- 
adequately in  terras  of  the  standards  referred  to  above,  and  must  then 
have  been  recommended  as  a drop  by  his  instructor  and  at  each  of  several 
ascending  levels  of  review.  There  may  have  been  considerable  differences 
in  the  subjective  judgments  that  formed  the  basis  for  these  recommenda- 
tions. 

AFAM  Time  to  Completion 

The  AFAM  times  for  CMI  and  IMI  students  were  quite  similar,  ranging 
from  31%  to  33%  of  the  times  required  by  Cl  students.  The  average  vari- 
ability within  Cl  cells  was  smaller  than  the  average  variability  within 
the  CMI  and  IMI  cells,  as  might  be  expected,  but  the  F ratios  were  quite 
large.  Separate  analyses  of  CMI  and  IMI  students  indicated  no  reliable 
differences  between  these  two  training  methods. 

In  the  overall  analyses,  there  were  statistically  reliable  Training- 
Met  ho  d-by-  Week  interactions  for  both  Navy  and  Marine  students.  This  was 
the  result  of  a holiday  which  led  to  an  accelerated,  4-day  course  for 
the  Cl  groups  that  were  trained  that  week.  Week-by-week  means  are  shown 
in  Appendix  E.  Aptitude  effects  were  not  statistically  reliable  in  the 
overall  analysis  of  Marine  students,  but  they  were  reliable  in  the  analysis 
that  was  limited  to  CMI  and  IMI  students. 
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TABLE  1 


PERFORMANCE  OF  NAVY  STUDENTS 


Training  Method 

CMI 

IMI 

Cl 

Drops  (%) 

5.5 

6.3 

2.1 

AFAM  Time  (hrs)* * 

10.5 

10.3 

33.6 

AFAM  % Correct* 

96.4 

95.3 

89.7 

AMFU  Time  (hrs)* 

39.7 

38.3 

73.2 

AMFU  % Correct* 

94.0 

93.5 

87.6 

Shop  Grade* 

89.8 

87.0 

93.3 

Attitude  I 

2.7 

2.7 

2.7 

Attitude  II 

2.7 

2.8 

- 

Delayed  Test  Z Correct 

78.8 

75.3 

74.7 

"A”  School  Grade 

78.4 

77.0 

77.7 

Note — 

*Both  CMI  and  IMI  differ  from  Cl,  p <.01 
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TABLE  2 


PERFORMANCE  OF  MARINE  STUDENTS 


Training  Method 

CMI 

IMI 

Cl 

Drops  (%) 

9.2 

7.0 

7.5 

AFAM  Time  (hrs)* * 

11.7 

10.3 

34.4 

AFAM  % Correct  * 

95.3 

94.1 

91.0 

AMFU  Time  (hrs)* 

41.0  ‘ 

41.3 

70.3 

AMFU  % Correct* 

93.3 

92.2 

87.6 

Shop  Grade* 

89.8 

88.2 

93.6 

Attitude  I 

2.9 

2.8 

2.7 

Attitude  II 

3.0 

2.9 

Delayed  Test  % Correct 

78.1 

75.5 

69.8 

"A"  School  Grade 

78.2 

76.2 

77.0 

Note — 

*Both  CMi  and  IMI  differ  from  Cl,  p <.01 
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AFAM  Percentage  Correct 


All  groups  had  scores  above  89%  on  this  variable.  The  CMI  and  IMI 
scores  ranged  from  three  to  seven  percentage  points  higher  than  those 
of  the  Cl  students.  In  all  cases,  the  differences  were  statistically 
reliable.  It  should  be  remembered,  though,  that  the  CMI  and  IMI  scores 
do  not  reflect  the  improvement  that  came  about  as  the  result  of  remediation 
following  the  initial  administration  of  the  end-of-course  examination. 

For  a nonsystematic  sample  of  36  CMI  students,  the  substitution  of  scores 
on  tests  following  remediation  for  scores  on  those  portions  of  the  end- 
of-course  examination  that  prompted  the  remediation  led  to  an  increase 
from  97.7%  to  99.0%  in  AFAM  and  from  96.1%  to  99.0%  in  AMFU  (the  original 
scores  for  this  sample  were  somewhat  higher  than  those  for  the  total  CMI 
sample) . 

The  analysis  of  the  data  from  Navy  students  was  complicated  by  two 
statistically  significant  interactions.  One  was  a three-way  interaction 
that  seemed  to  arise  from  an  Aptitude-by-Week  interaction  (for  which  there 
was  no  obvious  explanation)  within  the  Cl  method  that  was  not  present 
within  the  other  methods.  The  other  was  a Training-Method-by- Aptitude 
interaction  that  was  due  to  a larger  difference  between  medium  and  high 
aptitude  students  for  the  Cl  method  than  for  the  CMI  and  IMI  methods. 

At  least  part  of  this  interaction  was  probably  the  result  of  a simple 
ceiling  effect,  since  a difference  as  large  as  that  found  for  the  Cl  method 
would  have  required  scores  over  100%  in  both  the  CMI  and  IMI  methods. 

A similar  but  less  pronounced  pattern  was  found  for  the  Marine  students. 

The  latter  fell  short  of  statistical  significance. 

AMFU  Time  to  Completion 


Again,  the  times  for  the  CMI  and  IMI  students  were  quite  similar, 
but  in  this  case  they  ranged  from  54%  to  60%  of  the  times  required  by 
the  Cl  students.  As  was  the  case  with  AFAM  times,  variability  within 
the  Cl  cells  was  smaller,  on  the  average,  than  variability  within  the 
CMI  and  IMI  cells,  but  the  F ratios  were  quite  large.  Separate  analyses 
of  CMI  and  IMI  students  indicated  no  reliable  difference  between  these 
two  training  methods. 

There  was  a reliable  Aptitude-by-Week  interaction  for  the  overall 
analysis  on  Navy  students  that  seemed  to  arise  from  a different  pattern, 
over  weeks,  for  low  aptitude  students  than  for  middle  or  high  aptitude 
students,  but  for  which  there  was  no  obvious  explanation.  This  interaction 
was  not  reliable  for  Marine  students,  nor  was  it  reliable  when  the  analysis 
was  limited  to  CMI  and  IMI  students. 

AMFU  Percentage  Correct 

Again,  all  groups  had  fairly  high  scores,  but  in  each  case  they  were 
slightly  lower  than  those  obtained  in  the  AFAM  course.  As  a result,  the 
differences  between  the  students  in  the  individually  paced  courses  and 
the  students  in  the  Cl  course  were  roughly  the  same  as  those  found  pre- 
viously. Again,  all  were  statistically  reliable. 
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Shop  Grade 


This  Is  the  only  variable  on  which  there  were  reliable  differences 
favoring  the  Cl  group.  The  largest  of  these  differences  was  approximately 
six  percentage  points.  For  the  Marines  there  was  a statistically  sig- 
nificant Training-Method-by-Week  interaction  that  was  largely  the  result 
of  fluctuations  in  the  Cl  group  that  were  completely  out  of  phase  with 
those  in  the  CMI  and  IMI  groups.  Part  of  the  differences  between  methods, 
as  well  as  the  interaction,  may  have  been  due  to  differences  in  the  grading 
standards  used  by  the  relatively  few  instructors  who  scored  this  work. 

The  difference  between  CMI  and  IMI  students  was  statistically  reliable 
for  the  Navy  students,  but  not  for  the  Marine  students.  The  differences 
between  levels  of  aptitude  fell  short  of  statistical  significance  for 
the  Marine  students. 

Attitude  I 


There  were  no  reliable  differences  between  methods  on  this  variable. 

For  all  groups,  attitudes  toward  the  assigned  training  method  were  somewhat 
closer  to  favorable  than  to  neutral.  Responses  to  specific  questions 
can  be  found  in  Appendix  C. 

Attitude  II 

There  were  no  reliable  differences  on  this  variable,  not  even  in 
aptitude.  It  is  interesting  that  these  students  indicated  at  least  a 
mild  preference  for  individually-paced  instruction,  even  though  they  were 
no  more  favorable  than  the  Cl  group  on  Attitude  I.  Responses  to  specific 
questions  can  be  found  in  Appendix  C. 

Delayed  Retention 

In  all  cases  the  CMI  and  EMI  students  had  scores  that  were  higher 
than  those  of  the  Cl  students,  but  only  in  the  case  of  the  CMI  and  Cl 
Marines  was  the  difference  reliable.  The  average  difference  between  the 
individualized  groups  and  the  Cl  groups  on  the  delayed  tests  was  within  .7 
percentage  points  of  the  average  difference  found  on  the  end-of-course 
tests,  a finding  which  lends  no  support  to  the  fear  that  material  which 
is  learned  rapidly  will  also  be  forgotten  rapidly.  The  failure  to  find 
more  reliable  differences  on  this  variable  was  probably  due  in  part  to 
the  large  variability  within  groups.  For  the  Marines,  even  aptitude  levels 
failed  to  differ  reliably. 

In  general,  these  scores  were  15  to  20  percentage  points  lower  than 
the  corresponding  scores  on  the  end-of-course  tests.  Part  of  this  loss 
was  probably  due  to  a lack  of  involvement  on  the  part  of  some  students . 

Some  students  who  had  obviously  not  tried  were  retested  and  made  substan- 
tially higher  scores.  Previous  research  in  which  the  retention  testing 
was  monitored  by  the  instructors  who  originally  taught  the  material  indi- 
cated a much  smaller  loss . 
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Class  A School  Grade 


There  were  no  reliable  differences  between  the  various  groups  on  this 
variable. 

Costs 


Although  costs  are  not  results  in  the  same  sense  as  the  preceding 
criteria,  they  are  one  of  the  most  important  elements  of  an  evaluation. 

At  the  outset  it  was  felt  that  many  of  these  costs  could  simply  be  recorded 
as  the  project  went  along,  but  it  soon  became  obvious  that  most  of  the 
costs  recorded  in  this  manner  would  be  contaminated  by  factors  that  were 
either  accidental  or  associated  with  the  unique  problems  of  an  initial 
developmental  effort.  In  most  cases  informed  estimates  of  cost  will  pro- 
vide a more  valid  basis  for  judging  the  long-term  cost  effectiveness  of 
the  present  system,  or  for  extrapolations  to  other  systems. 

Preparation  of  Instructional  Material 

This  cost  will  be  roughly  the  same  as  the  cost  of  preparing  con- 
ventional programmed  instruction.  Estimates  of  100  manhours  per  hour 
of  instruction  are  fairly  common.  It  will  probably  be  possible  to  prepare 
material  somewhat  more  rapidly  because  of  the  reliance  on  lean  programs 
coupled  with  systematic  remediation,  but  this  advantage  will  probably 
be  more  than  offset  by  the  need  for  preparing  multiple  forms  of  the 
end-of-module  tests. 

Coding 

The  effort  involved  in  coding  a module  for  the  computer  will  vary 
greatly  as  a function  of  module  content  and  the  instructional  strategy 
that  is  followed  in  teaching  the  module.  Some  modules,  for  example  can 
be  tested  by  means  of  very  short  tests  (even  one-item  tests  in  extreme 
cases),  and  performance  following  remediation  can  be  tested  by  parallel 
versions  of  the  same  test.  In  other  cases,  very  long  tests  will  be  needed 
to  cover  specific  areas  within  the  module,  and  provisions  must  be  made 
for  selecting  from  among  a number  of  alternative  forms  of  remediation. 

The  modules  included  in  these  courses  were  moderately  complex,  and 
preparation  for  the  computer  probably  required  an  average  of  from  5 to 
10  manhours  each.  This  includes  coding,  card  punching,  and  debugging. 

ADP 

One  of  the  major  distinctions  between  the  CMI  and  IMI  systems  is  the 
cost  of  ADP  equipment  and  personnel.  Unfortunately,  the  actual  costs 
of  the  two  systems  were  almost  completely  confounded  during  the  period 
of  the  evaluation.  Even  had  they  not  been,  it  would  have  been  difficult 
to  arrive  at  costs  that  could  be  used  as  a realistic  basis  for  extrapo- 
lation, since  use  fluctuated  widely  during  the  period  covered.  There 
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also  tended  to  be  an  under  utilization  of  the  system  for  training  purposes 
because  of  the  developmental  nature  of  the  program. 

As  part  of  the  information  required  for  the  procurement  of  a Navy 
computer  to  be  used  in  support  of  CMI,  the  Computer  Center  at  the  Naval 
Air  Technical  Training  Center,  Memphis,  developed  estimated  costs  for 
various  numbers  of  students.  These  costs  covered  the  computer  itself, 
terminals,  and  both  military  and  civilian  ADP  personnel.  In  general, 
the  cost  per  student  decreased  with  increases  in  the  total  number  of 
students.  For  a 7-hour  training  day  the  estimated  cost  was  $.75  per  hour 
for  500  students,  and  about  $.20  per  hour  for  5000  students.  If  a like 
number  of  students  were  added  in  a second  shift,  the  cost  per  student 
hour  would  be  reduced  by  at  least  40%.  Cost  estimates  for  computer  hard- 
ware and  personnel  to  support  the  two  courses  in  an  operational  setting 
have  been  reported  in  a somewhat  different  format  by  Mayo  (1973,  1974). 

Similar  estimates  were  not  developed  for  the  IMI  system,  but  the  cost 
would  be  fairly  substantial.  In  fact,  the  IMI  system  used  during  the 
evaluation  was  actually  a less  elaborate  CMI  system. 

Instructors 


During  the  period  of  the  evaluation,  the  student  to  instructor  ratio 
was  approximately  10:1  in  both  the  CMI  and  IMI  learning  centers.  This 
was  about  the  same  as  the  ratio  that  existed  in  the  conventional  course. 
However,  there  were  several  respects  in  which  the  CMI  system  being  used 
during  the  evaluation  differed  from  the  CMI  system  that  is  being  imple- 
mented on  a general  basis  at  the  Naval  Air  Technical  Training  Center, 
Memphis . 

Serious  problems  were  encountered  in  running  the  original  version 
of  the  CMI  system.  Since  there  was  not  enough  time  to  rectify  these 
problems  through  major  modifications  of  the  software,  it  was  decided  to 
mitigate  their  effects  by  having  the  Instructors  duplicate,  or  assume 
entirely,  certain  functions  that  had  originally  been  allocated  solely 
to  the  computer.  Ihese  additional  duties  accounted  for  about  a third 
of  the  instructor  time  spent  in  the  CMI  learning  centers. 

At  the  present  time  there  are  several  80-student  learning  centers 
in  the  Basic  Electricity  and  Electronics  portion  of  the  Avionics  Technician 
course  that  are  being  run  under  CMI  by  five  instructors.  This  more  favor- 
able student-instructor  ratio  may  be  due  in  part  to  the  fact  that  students 
interact  with  the  computer  less  frequently  in  this  course.  However,  the 
major  factors  are  probably  the  use  of  single,  sequential  assignments  and 
a smaller,  less  expensive,  classroom  terminal  that  provides  neai>- instan- 
taneous responses.  These  features  eliminate  the  need  for  the  instructor 
to  monitor  the  answer  sheets  and  lesson  guides,  or  to  provide  remedial 
assignments.  Some  incidental  data  on  instructor  attitudes  toward  CMI 
can  be  found  in  Appendix  C. 
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DISCUSSION  AND  CONCLUSIONS 


There  are  three  main  points  that  should  be  stressed  concerning  the 
outcome  of  this  evaluation,  all  of  which  have  already  been  touched  on 
briefly.  The  first  is  the  rather  arbitrary  nature  of  the  CMI  system  that 
was  used  during  the  evaluation,  the  second  is  the  lack  of  differences 
between  CMI  and  IMI  in  terms  of  training  effectiveness,  and  the  third 
is  the  efficiency  of  these  training  systems  relative  to  alternative  train- 
ing systems. 

This  evaluation  should  not  be  interpreted  as  a definitive  evaluation 
of  CMI.  It  is  an  evaluation  of  one  particular  approach  to  CMI,  at  one 
particular  stage  of  its  development,  and  in  one  particular  training  con- 
text. The  system  developed  under  this  project  was  the  first  attempt  to 
design  a system  for  this  type  of  application,  and  it  suffered  from  many 
problems  of  the  kind  that  should  be  expected  in  any  initial  development 
effort.  Although  the  system  afforded  capabilities  that  were  never  used, 
it  lacked  others  that  would  have  Improved  results,  simply  because  neither 
time  nor  resources  were  available  for  all  the  needed  modifications.  The 
new  classroom  terminal  now  In  use,  for  example,  provides  an  efficient 
solution  to  many  of  the  problems  that  were  encountered  during  the  initial 
implementation  of  the  system  in  the  schools,  but  this  terminal  and  the 
software  needed  to  run  it  were  not  available  in  time  to  be  used  in  the 
evaluation.  There  have  been  many  Improvements  in  the  system  since  its 
inception,  and  there  is  no  reason  to  doubt  that  there  will  be  substantial 
improvements  in  the  years  to  come.  Nevertheless,  the  system  that  existed 
at  the  time  of  the  evaluation  proved  to  be  highly  effective  for  training. 

The  IMI  system  was  just  as  good  as  the  CMI  system  in  terms  of  training 
effectiveness,  and,  in  view  of  the  similarity  of  the  two  systems  from 
the  student’s  point  of  view,  any  other  finding  would  have  been  surprising. 
Under  such  circumstances  it  might  seem  reasonable  to  conclude  that  the 
level  of  computer  support  provided  by  the  IMI  system  is  a more  reasonable 
choice  than  the  level  of  computer  support  provided  by  the  CMI  system. 

To  justify  the  contrary,  it  is  necessary  to  go  beyond  the  data  provided 
by  the  formal  evaluation. 

The  argument  for  the  higher  level  of  support  is  based  upon  two  main 
assumptions.  First,  if  the  instructors  can  be  relieved  of  the  jobs  of 
tracking  the  students  through  the  course,  evaluating  answer  sheets,  and 
making  remedial  assignments,  then  It  should  be  possible  to  effect  sub- 
stantial increases  in  the  student-instructor  ratio.  Some  of  the  reasons 
why  this  could  not  be  done  during  the  evaluation  have  already  been  Indi- 
cated, as  have  the  reasons  for  believing  that  some  of  the  difficulties 
which  existed  during  the  evaluation  have  subsequently  been  corrected  by 
system  modifications. 
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The  second  assumption  is  that  a moderately  high  level  of  computer 
support  will  be  required  to  manage  any  large  scale  system  of  individualized 
instruction.  Such  a level  was  certainly  provided  for  IMI  during  the 
evaluation.  In  the  kind  of  situation  under  consideration  here,  it  is 
also  reasonable  to  assume  that  there  will  be  decreases  in  the  marginal 
cos ts  of  increased  computer  utilization.  Estimates  of  these  costs  and 
cost  avoidances  were  used  by  the  Computer  Center  at  the  Naval  Air  Technical 
Training  Center,  Memphis,  to  justify  the  higher  level  of  support  on  the 
grounds  that  the  decrease  in  the  cost  of  instructors  would  more  than  offset 
the  increased  costs  of  computer  support  required  to  score  all  answer 
sheets,  make  all  assignments,  and  track  the  student  through  the  course. 

The  higher  level  support  also  provides  the  basis  for  a number  of  addi- 
tional capabilities  that  would  not  be  available  with  the  lower  level  sup- 
port. For  example,  it  provides  a greater  flexibility  in  tailoring  both 
course  content  and  training  methods  to  the  needs  and  capabilities  of 
individual  students.  It  also  provides  the  data  required  for  the  optimum 
allocation  of  scarce  instructional  resources,  the  rapid  identification 
of  students  who  are  having  difficulties,  and  the  systematic  improvement 
of  instructional  materials. 

For  comparisons  other  than  those  between  CMI  and  IMI,  it  will  be  help- 
ful to  attach  at  least  crude  monitary  values  to  reductions  in  training 
time.  The  students  and  instructors  who  are  tied  down  in  formal  school 
training  are  not  immediately  available  for  use  in  meeting  the  Navy's 
operational  commitments.  Any  reduction  in  the  number  of  men  in  these 
categories  (assuming  no  reduction  in  the  number  or  proficiency  of  grad- 
uates) should  provide  a comparable  reduction  in  the  total  number  of  men 
required  by  the  Navy.  Training  time  can  be  converted  directly  to  student 
and  instructor  man-years,  and  reductions  in  these  man-years  should  represent 
a source  of  real  savings.  In  the  following  discussion,  the  estimated 
savings  are  based  on  annual  personnel  costs  as  determined  by  the  Comp- 
troller of  the  Navy  at  the  time  of  the  evaluation.  These  were  taken  to 
be  $5890  for  students  (E-3),  and  $9697  for  instructors  (E-6). 

Before  turning  to  the  savings  provided  by  CMI,  it  might  be  noted  that 
considerable  savings  were  provided  by  the  revision  of  training  objectives, 
one  of  the  by-products  of  the  research  project.  Even  if  there  had  been 
no  individual  pacing  of  instruction,  the  revision  of  the  conventional 
courses  to  meet  these  new  objectives  provided  a 50%  reduction  in  training 
time,  and,  therefore,  in  the  number  of  students  under  training  at  any 
given  time.  With  an  average  input  of  300  students  per  week,  which  is 
quite  conservative,  this  would  have  provided  a cost  reduction  of  approx- 
imately six  million  dollars  a year. 

The  individual  pacing  of  instruction,  whether  by  CMI  or  IMI,  provided 
an  additional  reduction  in  average  training  time  of  about  50%.  This, 
in  turn,  provided  an  additional  cost  reduction  of  approximately  three 
million  dollars  a year.  This  represents  almost  twice  the  total  cost  of 
the  research  project,  including  the  salaries  of  military  personnel,  and 
is  about  five  times  the  annual  cost  of  supporting  a CMI  system  in  these 
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particular  schools,  including  all  ADP  costs  and  the  cost  of  special  in- 
structional equipment. 

« 

In  conclusion,  it  might  be  worthwhile  to  compare  both  the  cost  and 
benefits  of  this  CMI  system  with  similar  data  that  have  been  reported 
for  CAI  systems.  Probably  the  most  meaningful  comparisons  can  be  made 
with  applications  of  CAI  within  military  training  situations.  One  of 
the  first  (IBM,  1968)  reported  a reduction  in  training  time  of  only  11.5%. 
With  revisions  of  this  same  material,  the  savings  increased  to  20.1% 
(Longo,  1969).  Finally,  as  more  material  was  programmed,  the  savings 
increased  to  better  than  35%  (Longo,  1972).  All  of  these  studies  were 
done  using  the  IBM  1500  system  to  teach  the  early  portions  of  an  Army 
course  in  electroncis.  In  another  study  in  which  the  1500  system  was 
used  by  the  Navy  to  teach  the  early  portions  of  a course  in  electronics, 
a savings  of  about  45%  was  reported  (Ford,  Slough,  and  Hurlock,  1972). 
Finally,  studies  in  which  the  Lincoln  Training  System  was  used  to  teach 
an  early  portion  of  an  Air  Force  course  in  electronics  report  savings 
that  range  from  37%  (Lincoln  Laboratory,  MIT,  1972)  to  50%  (Lincoln  Lab- 
oratory, MIT,  1973;  Training  Research  Applications  Branch,  Keesler  AFB, 
1973). 

On  the  surface,  the  overall  reduction  in  training  time  found  in  this 
study  would  seem  to  be  competitive  with  the  reductions  reported  for  CAI, 
but,  as  is  usually  the  case,  there  are  complications.  On  the  one  hand, 
the  control  for  variations  in  training  objectives  was  somewhat  more 
stringent  in  this  study  than  it  was  in  certain  of  the  CAI  studies.  On 
the  other,  it  is  quite  possible  that  the  materials  taught  in  this  study 
provided  somewhat  greater  opportunities  for  reductions  in  training  time 
than  did  the  material  on  electronics  taught  in  the  CAI  studies.  Relative 
advantages  or  disadvantages  of  CAI,  in  contrast  to  CMI,  will  not  be 
apparent  until  comparisons  are  made  in  terms  of  alternative  training 
treatments,  instructor  to  student  ratios,  types  of  instructional  material, 
ease  of  instructional  modification,  and  other  pertinent  variables.  The 
decision  to  select  any  instructional  procedure,  including  CMI  or  CAI, 
dependsupon  many  factors  including  training  objectives,  criticality  of 
the  training  and  other  situational  variables  which  may  or  may  not  enter 
into  direct  dollar  cost.  Because  of  these  considerations,  it  is  difficult 
to  generalize  about  the  relative  values  of  CMI  and  CAI. 

In  terms  of  cost  per  training  hour,  CMI  of  the  type  described  here 
seems  to  enjoy  a clear  advantage  over  CAI.  The  costs  of  the  system,  even 
in  its  limited  application  to  the  two  courses  used  in  the  evaluation, 
are  less  than  the  costs  that  have  been  projected  for  CAI  in  the  next  few 
years.  (See  Ford,  Slough,  andJlurlock,  1972,  for  a discussion  of  CAI 
costs.)  The  costs  for  a large-scale  CMI  implementation  of  the  kind  that 
has  been  initiated  at  the  Naval  Air  Technical  Training  Center,  Memphis, 
are  far  less. 
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THE  IMMEDIATE  FUTURE 


At  the  time  this  report  is  being  prepared,  action  pertaining  to  CMI 
is  taking  place  on  two  fronts  within  the  Navy.  The  first  of  these  is 
the  use,  improvement,  and  expansion  of  the  present  system  by  the  Navy 
Technical  Training  Command.  CMI  is  being  used  currently  at  the  Naval 
Air  Technical  Training  Center,  Memphis,  in  the  Aviation  Fundamentals  Course, 
Class  P (a  successor  to  the  two  courses  described  in  this  report),  the 
Basic  Electricity  and  Electronics  Course,  Class  P,  and  in  the  Aviation 
Machinist's  Mate  J (Jet)  Course,  Class  A.  The  Memphis  computer  is  also 
being  used  to  support  CMI  in  Basic  Electricity  and  Electronics  Course,  Class 
P,  at  the  Service  School  Command,  San  Diego.  The  current  on-board  load 
under  CMI  is  approximately  2500  students.  Plans  have  been  made  for  a 
sizeable  expansion  of  the  system  to  other  courses,  both  in  Memphis  and 
in  remote  locations. 

The  second  front  in  which  CMI  is  being  applied  is  that  of  additional 
research.  A project  designed  to  test  the  feasibility  of  CMI  as  a means 
for  providing  an  entire  technical  training  sequence  tailored  to  the  re- 
quirements of  specific  billets  in  the  fleet  has  recently  been  completed. 
Another  ongoing  project  is  designed  to  test  the  feasibility  of  using 
minicomputers  for  applying  CMI  to  shipboard  personnel. 

In  addition  to  these  two  major  CMI  efforts  a number  of  Exploratory 
Development  projects  are  oriented  toward  an  improved  technology  of  in — 
dividualized  instruction.  The  results  of  these  efforts  should  help  to 
increase  the  efficiency  of  CMI. 
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APPENDIX  A 


HISTORICAL  ASPECTS  OF  THE  CMI  PROJECT 

The  CMI  project  was  initiated  in  fiscal  year  1968  as  a joint  under- 
taking  of  the  Chief  of  Naval  Air  Technical  Training  and  the  Navy  Training 
Research  Laboratory  Branch  Office,  Memphis.  Support  from  external  sources 
for  the  first  year  consisted  of  funds  provided  by  the  Assistant  Secretary 
of  Defense  (Manpower  and  Reserve).  Since  fiscal  year  1969,  the  primary 
support  of  the  CMI  project  has  come  in  approximately  equal  amounts  from 
the  Navy's  Advanced  Development  Objective  43-03X  (Education  and  Training) 
and  from  the  Chief  of  Naval  Technical  Training.  ADO  43-03X  has  supported 
research  and  development  aspects  of  the  project.  The  Chief  of  Naval 
Technical  Training  has  funded  development  of  instructional  material, 
renovation  of  spaces  for  computer-based  instruction,  and  computer  support 
in  the  on-going  training  situation. 

In  terms  of  actual  dollar  amounts,  $866K  was  received  from  ADO  43-03X. 
The  Chief  of  Naval  Technical  Training  provided  $210K,  plus  the  services 
of  approximately  six  Navy  petty  officers  for  the  duration  of  the  project. 
The  Naval  Air  Systems  Command  provided  $130K  for  training  equipment,  and 
the  Assistant  Secretary  of  Defense  (Manpower  and  Reserve),  $70K  in  the 
initial  phase  of  the  project.  The  Naval  Air  Technical  Training  Center, 
Memphis,  provided  two  Navy  petty  officers  for  most  of  the  project,  plus 
assistance  in  the  derivation  of  training  objectives.  NATTC  also  made 
available  students  in  the  numbers  needed  throughout  the  development  and 
evaluation  of  the  CMI  system.  The  Naval  Air  Station,  Memphis,  provided 
one  civilian  computer  analyst  during  most  of  the  project,  and  several 
computer  and  terminal  operators  during  the  latter  part  of  the  project. 
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APPENDIX  B 


Content  of  AFAM  and  AMFU  Courses 


AFAM  Modules 

1.  The  Military  Aircraft  Designation  System 


2. 

Basic  Theory  of  Flight  and  Aircraft 
clature 

Nomen' 

3. 

Aircraft  Handling  Crews,  Securing  Devices 
and  Safety  in  Line  Operations 

4. 

Aviation  Support  Equipment 

5. 

Aviation  Rating  Familiarization  or  Marine 
Aviation  Occupational  Fields 

6. 

Aircraft  Carriers 

7. 

Aircraft  Firefighting 

8. 

Naval  Aviation  Organization  or  Marine 
Aviation  Organization 

9. 

Standard  Aircraft  Taxi  Signals 

10. 

Basic  Aircraft  Systems 

11. 

Aircraft  Cleaning 

12. 

Aviation  Fuels,  Oils,  and  Hydraulic 

Fluid 

31 


AMFU  Modules 


1. 

Introduction  to  the  Naval  Aviation  Maintenance 
Program 

2. 

The  Work  Unit  Code  Manual 

3. 

Maintenance  Requirements  Cards 

4. 

Corrosion 

5. 

Mechanics  of  Heat  and  Gases,  Static  Electricity 
and  Basic  Hydraulics 

6. 

Addition,  Subtraction,  Multiplication,  and 
Division  of  Fractions 

7. 

Addition,  Subtraction,  Multiplication,  and 
Division  of  Decimals 

8. 

The  Support  Action  Form 

9. 

The  Maintenance  Action  Form  - Single  Copy 

10. 

Aircraft  Hardware 

11. 

Wrenches 

12. 

Screwdrivers  and  Pliers 

13. 

Measuring  and  Marking  Tools  and  Drills 

14. 

Vices,  Files,  and  Hacksaws 

15. 

Punches,  Chisels,  and  Striking  Tools 

16. 

Maintenance  and  Operation  Manuals 

17. 

The  Maintenance  Action  Form  - Multi-Copy 

18. 

Torque  Wrenches 

19. 

Shop  I 

20. 

Shop  II 
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APPENDIX  C 


Student  Attitude  Questionnaire 

Questions  1 through  11  were  common  to  all  three  training  methods  and 
were  used  as  the  basis  for  the  Attitude  I scores.  For  questions  1,  3, 

4,  7,  9,  10,  and  11,  the  last  alternative  was  assumed  to  be  indicative 
of  a favorable  attitude.  For  questions  2,  5,  6,  and  8,  the  first  alter- 
native was  assumed  to  be  indicative  of  a favorable  attitude. 

Questions  12  through  25  were  common  to  CMI  and  IMI.  Questions  12 
through  19  were  used  as  the  basis  for  the  Attitude  II  scores.  For  ques- 
tions 12,  15,  18,  and  19,  the  last  alternative  was  assumed  to  be  indic- 
ative of  a favorable  attitude.  For  questions  13,  14,  16,  and  17,  the  first 
alternative  was  assumed  to  be  indicative  of  a favorable  attitude. 

The  percentages  under  the  treatment  columns  are  based  on  a nonsystematic 
sample  of  201  students  from  each  treatment. 
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L. 


Student  Questionnaire 


The  following  statements  apply  to  the  way  you  felt  while 
you  were  a student  in  the  AFAM/AMFU  courses.  We  are 
interested  in  your  opinion  on  each  of  these  statements.  Do 
not  hesitate  to  put  down  exactly  how  you  feel  about  each 
item.  Your  honest  opinion  will  be  most  helpful. 

Mark  on  your  answer  sheet  the  number  of  the  response  that 
most  nearly  represents  your  reaction  to  each  of  the  statements 
below.  Do  not  write  on  this  booklet. 

% Responses 


CMI 

IMI 

Cl 

I 

felt  challenged  to  do  my  best  work. 

1. 

Strongly  disagree 

1 

1 

2 

2. 

Disagree 

5 

2 

7 

3. 

Uncertain 

4 

6 

15 

4. 

Agree 

59 

63 

56 

5. 

Strongly  agree 

31 

27 

19 

I 

was  concerned  that  I might  not  be 

understanding  the  material. 

1. 

Strongly  disagree 

12 

9 

7 

2. 

Disagree 

42 

35 

30 

3. 

Uncertain 

11 

9 

17 

4. 

Agree 

29 

38 

34 

5. 

Strongly  agree 

6 

8 

12 

I 

felt  uncertain  as  to  my  performance 

relative  to  the  performance  of  others. 

1. 

All  the  time 

7 

8 

5 

2. 

Most  of  the  time 

14 

14 

9 

3. 

Some  of  the  time 

34 

38 

28 

4. 

Only  occasionally 

22 

22 

33 

5. 

Never 

22 

17 

25 

I felt  myself  just  trying  to  get 

through 

the 

courses  rather  than  trying  to 

learn. 

1. 

All  the  time 

1 

1 

1 

2. 

Most  of  the  time 

6 

4 

3 

3. 

Some  of  the  time 

13 

17 

14 

4. 

Only  occasionally 

28 

40 

27 

5. 

Never 

52 

38 

55 

37 


Cl 


CMI  IMI 

5.  I tried  to  learn  as  much  as  I could. 


1. 

All  the  time 

63 

55 

54 

2. 

Most  of  the  time 

29 

40 

37 

3. 

Some  of  the  time 

5 

3 

6 

4. 

Only  occasionally 

2 

0 

2 

5. 

Never 

0 

2 

1 

I felt  pushed. 

1. 

Strongly  disagree 

25 

18 

27 

2. 

Disagree 

32 

26 

47 

3. 

Uncertain 

10 

14 

10 

4. 

Agree 

19 

31 

13 

5. 

Strongly  agree 

13 

10 

3 

The 

material  was  easy  to  learn. 

1. 

Strongly  disagree 

1 

2 

3 

2. 

Disagree 

14 

16 

18 

3. 

Uncertain 

16 

23 

18 

4. 

Agree 

54 

49 

51 

5. 

Strongly  agree 

14 

9 

9 

The 

material  was  difficult  to  remember. 

1. 

Strongly  disagree 

12 

6 

8 

2. 

Disagree 

64 

57 

54 

3. 

Uncertain 

8 

18 

10 

4. 

Agree 

13 

17 

24 

5. 

Strongly  agree 

2 

1 

3 

9.  I felt  frustrated  by  the  way  these  courses 
were  run. 


1. 

Strongly  agree 

5 

5 

5 

2. 

Agree 

12 

13 

6 

3. 

Uncertain 

11 

13 

12 

4. 

Disagree 

46 

47 

55 

5. 

Strongly  disagree 

26 

21 

21 

38 


CMI 

IMI 

Cl 

10. 

The 

material  was  presented  effectively. 

1. 

Strongly  disagree 

1 

2 

3 

2. 

Disagree 

5 

4 

8 

3. 

Uncertain 

7 

7 

4 

4. 

Agree 

62 

61 

64 

5. 

Strongly  agree 

25 

25 

20 

11. 

I felt  that  too  much  was  expected  of  me. 

1. 

Strongly  agree 

2 

1 

2 

2. 

Agree 

6 

6 

1 

3. 

Uncertain 

9 

8 

6 

4. 

Disagree 

54 

56 

60 

5. 

Strongly  disagree 

28 

28 

30 

12. 

I felt  I could  work  at  my  own  pace. 

1. 

Strongly  disagree 

5 

7 

2. 

Disagree 

15 

12 

3. 

Uncertain 

4 

8 

4. 

Agree 

48 

47 

5. 

Strongly  agree 

27 

25 

13. 

The 

learning  was  too  mechanical. 

1. 

Strongly  disagree 

20 

23 

2. 

Disagree 

58 

60 

3. 

Uncertain 

11 

10 

4. 

Agree 

7 

4 

5. 

Strongly  agree 

3 

2 

14. 

This  type  of  instruction  made  me  feel 
quite  tense. 

1. 

Strongly  disagree 

16 

12 

2. 

Disagree 

47 

45 

3. 

Uncertain 

13 

13 

4. 

Agree 

16 

22 

5. 

Strongly  agree 

7 

8 

39 

CMI 

IMI 

15. 

I 

would  prefer  this  type  of  in- 

struction  over  traditional  instruction. 

1. 

Strongly  disagree 

5 

7 

2. 

Disagree 

14 

12 

3. 

Uncertain 

15 

15 

4. 

Agree 

25 

37 

5. 

Strongly  agree 

41 

29 

16. 

I 

tried  to  learn  as  quickly  as  I could 

in 

this  course. 

1. 

All  the  time 

57 

53 

2. 

Most  of  the  time 

36 

37 

3. 

Some  of  the  time 

4 

6 

4. 

Only  occasionally 

1 

3 

5. 

Never 

2 

0 

17. 

I 

felt  that  I could  have  learned  just 

as 

much  without  taking  so  many  tests 

during  the  courses . 

1. 

Strongly  disagree 

27 

31 

2. 

Disagree 

49 

41 

3. 

Uncertain 

7 

15 

4. 

Agree 

13 

10 

5. 

Strongly  agree 

4 

2 

18. 

In 

view  of  the  amount  I learned,  I 

would  say  this  type  of  instruction  is 
superior  to  traditional  instruction. 

1. 

Strongly  disagree 

3 

5 

2. 

Disagree 

13 

7 

3. 

Uncertain 

19 

18 

4. 

Agree 

36 

45 

5. 

Strongly  agree 

29 

24 

19. 

I 

would  prefer  subsequent  courses  to  be 

this  type  of  instruction  rather  than 
traditional  instruction. 


1. 

Strongly  disagree 

2 

5 

2. 

Disagree 

17 

13 

3. 

Uncertain 

16 

22 

4. 

Agree 

36 

40 

5. 

Strongly  agree 

28 

19 

40 


20. 


21. 


22. 


23. 


24. 


CMI  IMI 

The  instructor  was  available  whenever 
I needed  him. 


1. 

All  the  time 

75 

78 

2. 

Most  of  the  time 

21 

17 

3. 

Some  of  the  time 

2 

4 

4. 

Only  occasionally 

2 

0 

5. 

Never 

0 

0 

One 

of  the  main  reasons  that  I 

worked 

hard  during  the  day  was  to  avoid  oeing 

sent  to  night  school. 

1. 

Strongly  agree 

8 

6 

2. 

Agree 

14 

17 

3. 

Uncertain 

4 

5 

4. 

Disagree 

44 

46 

5. 

Strongly  disagree 

30 

25 

I felt  I would  be  sent  to  night 

school  no 

matter  how  hard  I worked  during 

the  day. 

1. 

Strongly  agree 

1 

3 

2. 

Agree 

5 

7 

3. 

Uncertain 

3 

8 

4. 

Disagree 

46 

44 

5. 

Strongly  disagree 

44 

37 

I could  have  learned  faster  if 

the  room 

had 

not  been  so  noisy. 

1. 

Strongly  agree 

4 

3 

2. 

Agree 

11 

12 

3. 

Uncertain 

9 

10 

4. 

Disagree 

50 

47 

5. 

Strongly  disagree 

26 

27 

There  were  occasions  when  I was 
carrel  with  nothing  to  do. 

at  my 

1. 

Very  often 

1 

1 

2. 

Often 

1 

1 

3. 

Some  of  the  time 

10 

3 

4. 

Only  occasionally 

34 

8 

5. 

Never 

54 

86 

41 


CMI  I MI 

25.  I could  have  learned  just  as  much  if  I 


had 

spent  less  time  in  class. 

1. 

Strongly  agree 

1 

2 

2. 

Agree 

6 

1 

3. 

Uncertain 

12 

12 

4. 

Disagree 

48 

51 

5. 

Strongly  disagree 

33 

33 
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APPENDIX  D 


ANOVA  Tables 


Navy:  AFAM  Time  (hrs) , All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

44446.96 

2 

22223.48 

1181.70* 

Aptitude  (A) 

330.87 

2 

165.43 

8.80* 

Week  (W) 

269.44 

6 

44.91 

2.39 

TM  X A 

631.97 

4 

40.16 

2.14 

TM  X W 

160.63 

12 

52.66 

2.80* 

A X W 

239.20 

12 

19.93 

1.06 

TM  X A X W 

399.29 

24 

16.64 

. 88 

error  within 

7917.46 

421 

18.81 

Note 

* p < . 01 
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Navy:  AFAM  Time  (hrs) , CMI  and  IMI  only 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

2.79 

1 

2.79 

.12 

Aptitude  (A) 

556.19 

2 

278.09 

11.82 

Week  (W) 

82.06 

6 

13.68 

.58 

TM  X A 

3.42 

2 

1.71 

.07 

TM  X W 

189.54 

6 

31.59 

1.34 

A X W 

423.34 

12 

35.28 

1.50 

TM  X A X W 

285.42 

12 

23.78 

1.01 

error  within 

7833.82 

333 

23.52 

Note 

* p <.01 
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Marine:  AFAM  Time  (hrs) , All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

26362.10 

2 

13181.05 

603.26* 

Aptitude  (A) 

48.25 

1 

48.26 

2.21 

Week  (W) 

243.53 

6 

40.59 

1.86 

TM  X A 

86.27 

2 

43.13 

1.97 

TM  X W 

469.63 

12 

55.80 

2.55* 

A X W 

127.45 

6 

21.24 

.97 

TM  X A X W 

371.13 

12 

30.93 

1.42 

error  within 

5353.17 

245 

21.  85 

Note — 

* p < .01 
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Marine:  AFAM  Time  (hrs) , CMI  and  IMI  only 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

68.62 

1 

68.62 

4.84 

Aptitude  (A) 

105.17 

1 

105.17 

7.41* * 

Week  (W) 

152.23 

6 

25.37 

1.79 

TM  X A 

4.37 

1 

4.37 

.31 

TM  X W 

165.67 

6 

27.61 

1.95 

A X W 

196.92 

6 

32.82 

2.31 

TM  X A X W 

149.58 

6 

24.93 

1.76 

error  within 

2142.37 

151 

14.19 

Note — 

* p < . 01 
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Navy:  Number  of  Errors  on  AFAM  Final  Examination, 

All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

15585.79 

2 

7792.89 

93.76* * 

Aptitude  (A) 

6844.90 

2 

3422.45 

41.18* 

Week  (W) 

1392.90 

6 

232.15 

2.79 

TM  X A 

2207.19 

4 

551.80 

6.64* 

TM  X W 

2078.27 

12 

173.19 

2.08 

A X W 

1441.89 

12 

120.16 

1.45 

TM  X A X W 

4423.99 

24 

184.33 

2.22* 

error  within 

34991.79 

421 

83.12 

Note — 

* p < . 01 
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Marine:  Number  of  Errors  on  AFAM  Final  Examination, 

All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

2259.42 

2 

1129.71 

15.80* * 

Aptitude  (A) 

2308.68 

1 

2308.68 

32.28* 

Week  (W) 

357.72 

6 

59.62 

. 83 

TM  X A 

244.28 

2 

122.14 

1.71 

TM  X W 

1135.95 

12 

94.66 

1.32 

A X W 

298.04 

6 

49.67 

. 69 

TM  X A X W 

757.55 

12 

63.13 

.88 

error  within 

17377.99 

243 

71.  51 

Note-- 

* p < .01 
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Navy:  AMFU  Time  (hrs) , All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

91548.19 

2 45774.09 

348.89* 

Aptitude  (A) 

18522.64 

2 

9261.32 

70.59* 

Week  (W) 

1679.89 

6 

279.98 

2.13 

TM  X A 

316.83 

4 

79.21 

. 60 

TM  X W 

2239.39 

12 

186.62 

1.42 

A X W 

4456.84 

12 

371.40 

2.83* 

TM  X A X W 

5448.76 

24 

227.03 

1.73 

error  within 

50832.56 

417a 

131.20 

Notes — 

* p < 

adf  and 

.01 

harmonic  mean 

adjusted 

because 

of  one 

cell  containing  single  observation 
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Navy:  AMFU  Time  (hrs) , CMI  and  IMI  only 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

110.88 

1 

110.88 

1. 13 

Aptitude  (A) 

13572.96 

2 

6786.48 

69.36* 

Week  (W) 

354.25 

6 

59.04 

.60 

TM  X A 

71.08 

2 

35.54 

.36 

TM  X W 

443.65 

6 

73.94 

.76 

A X W 

2237.53 

12 

186.46 

1.91 

TM  X A X W 

716.17 

12 

59.68 

.61 

error  within 

28282.60 

329a 

97.84 

Notes — 

*p  <.01 

adf  and 

harmonic  mean 

adjusted 

because  of 

one  cell 

containing  single  observation 


Marine:  AMFU  Time  (hrs) , All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

40259.31 

2 

20129.65 

143.20* * 

Aptitude  (A) 

2933.91 

1 

2933.91 

20.87* 

Week  (W) 

898.15 

6 

149.69 

1.06 

TM  X A 

526.95 

2 

263.47 

1.87 

TM  X W 

871.61 

12 

72.63 

.52 

A X W 

459.97 

6 

76.66 

.55 

TM  X A X W 

638.78 

12 

53.23 

1.87 

error  within 

34017.91 

242 

140.57 

Note — 

*p  < .01 
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Marine:  AMFU  Time  (hrs) , CMI  and  IMI  only 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

2.20 

1 

2.  20 

.02 

Aptitude  (A) 

2416.42 

1 

2416.42 

21.57* 

Week  (W) 

889.14 

6 

148.19 

1.32 

TM  X A 

310.07 

1 

310.07 

2.78 

TM  X W 

193.56 

6 

32.26 

.29 

A X W 

275.56 

6 

45.93 

.41 

TM  X A X W 

408.38 

6 

68.06 

.61 

error  within 

16692.72 

149 

112.03 

Note — 

*p  <.01 
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Navy:  Number  of  Errors  on  AMFU  Final  Examination, 

All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

35206.22 

2 

17603.11 

102.41* 

Aptitude  (A) 

20946.15 

2 

10473.07 

60.93* 

Week  (W) 

1455.64 

6 

242.61 

1.41 

TM  X A 

1534.78 

4 

383.70 

2.23 

TM  X W 

3482.45 

12 

290.20 

1.69 

A X W 

759.21 

12 

63.27 

.39 

TM  X A X W 

3108.39 

24 

129.52 

.75 

error  within 

72020.06 

419 

171.89 

Note— 

*p  < .01 
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Marine:  Number  of  Errors  on  AMFU  Final  Examination, 

All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

15769.63 

2 

7884.82 

42.40* * 

Aptitude  (A) 

4492.63 

1 

4492.63 

24.16* 

Week  (W) 

663.16 

6 

110.53 

.59 

TM  X A 

327.79 

2 

163.90 

. 88 

TM  X W 

2269.34 

12 

189.11 

1.02 

A X W 

674.91 

6 

112.49 

.60 

TM  X A X W 

1132.00 

12 

94.33 

.51 

error  within 

45371.72 

244 

185.95 

Note-- 

*p  < .01 


Navy:  Shop  Grade,  All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

2516.01 

2 

1258.01 

53.78* 

Aptitude  (A) 

404.16 

2 

202.08 

8.64* 

Week  (W) 

155.59 

6 

25.93 

1.11 

TM  X A 

62.62 

4 

15.65 

.67 

TM  X W 

538.87 

12 

44.91 

1.92 

A X W 

374.75 

12 

31.23 

1.34 

TM  X A X W 

512.31 

24 

21.35 

. 91 

error  within 

9848.21 

421 

23.39 

Note-- 

*p  < .01 
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Marine:  Shop  Grade,  All  Methods 


Source  SS 


Trng  Meth  (TM) 

1126.16 

Aptitude  (A) 

49.18 

Week  (W) 

154.91 

TM  X A 

90.16 

TM  X W 

510.62 

A X W 

161.47 

TM  X A X W 

349.98 

error  within 

4541.42 

df 

MS 

F 

2 

563.08 

30.38* * 

1 

49.18 

2.65 

6 

25.82 

1.39 

2 

45.08 

2.43 

12 

42.55 

2.30* 

6 

26.91 

1.45 

12 

29.16 

1.57 

245 

18.54 

Note-- 

*p  < .01 
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Navy:  Attitude  I Totals,  All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

30.29 

2 

15.15 

.41 

Aptitude  (A) 

414.17 

2 

207.08 

5.65* 

Week  (W) 

114.89 

6 

19.15 

.52 

TM  X A 

200.51 

4 

50.13 

1.37 

TM  X W 

513.24 

12 

42.77 

1.17 

A X W 

345.42 

12 

28.79 

.79 

TM  X A X W 

706.23 

24 

29.43 

1.37 

error  within 

10761.14 

328a 

36.59 

Notes — 

*p  <.01 

adf  and 

harmonic  mean 

adjusted 

because  of 

two  cells 

containing  single  observation 
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Marine:  Attitude  I Totals,  All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

114.71 

2 

57.35 

2.00 

Aptitude  (A) 

218.38 

1 

218.38 

7.63* 

Week  (W) 

373.93 

6 

62.32 

2.18 

TM  X A 

55.71 

2 

27.86 

.97 

TM  X W 

383.23 

12 

31.94 

1.12 

A X W 

186.58 

6 

31.10 

1.09 

TM  X A X W 

358.80 

12 

29.90 

1.05 

error  within 

6294.93 

220 

28.61 

Note— 

*p  < .01 
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Navy:  Attitude  II  Totals , CMI  and  IMI  only 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

28.78 

1 

28.78 

.87 

Aptitude  (A) 

21.32 

2 

10.66 

.32 

Week  (W) 

46.57 

6 

7.76 

.23 

TM  X A 

25.09 

2 

12.54 

.38 

TM  X W 

136.52 

6 

22.75 

.69 

A X W 

456.75 

12 

38.06 

1.15 

TM  X A X W 

197.69 

12 

16.47 

.50 

error  within 

6925.50 

252a 

33.05 

Note — 

df  and  harmonic  mean  adjusted  because  of  two  cells 
containing  single  observation 
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Marine:  Attitude  II  Totals,  CMI  and  IMI  only 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

12.76 

1 

12.76 

.56 

Aptitude  (A) 

8.78 

1 

8.78 

.39 

Week  (W) 

183.27 

6 

30.54 

1.35 

TM  X A 

80.87 

1 

80.87 

3.56 

TM  X W 

173.12 

6 

28.85 

1.27 

A X W 

398.40 

6 

66.40 

2.93 

TM  X A X W 

122.13 

6 

20.35 

.90 

error  within 

3085.92 

136 

22.69 
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Navy:  Number  of  Errors  on  Delayed  Tests,  All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

4771.74 

2 

2385.87 

1.81 

Aptitude  (A) 

13592.09 

2 

6796.04 

5.16* * 

Test  Form  (F) 

6705.40 

1 

6705.40 

5.09 

TM  X A 

2576.45 

4 

644.11 

.49 

TM  X F 

9303.10 

2 

4651.55 

3.53 

A X F 

873.87 

2 

436.93 

.33 

TM  X A X F 

3330.13 

4 

832.53 

.63 

error  within 

263653.69 

200 

1318.27 

Note — 

*p  <.01 
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Marine:  Number  of  Errors  on  Delayed  Tests,  All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

13516.43 

2 

6758.21 

5.51* * 

Aptitude  (A) 

8025.78 

2 

4012.89 

3.27 

Test  Form  (F) 

1180.60 

1 

1180.60 

.96 

TM  X A 

3937.53 

4 

984.38 

. 80 

TM  X F 

3790.61 

2 

1895.30 

1.  55 

A X F 

12.32 

2 

6.16 

. 01 

TM  X A X F 

9138.60 

4 

2284.65 

1.86 

error  within 

177875.69 

145 

1226.73 

Note — 

*p  <.01 
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Navy:  Class  A School  Grades,  All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

5287.31 

2 

2643.66 

CO 

CD 

• 

Aptitude  (A) 

183928.88 

2 

91964.44 

30.60* 

TM  X A 

16435.76 

4 

4108.94 

1.37 

error  within 

607076.00 

202 

3005.33 

Note — 

*p  <.01 
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Marine:  Class  A School  Grades,  All  Methods 


Source 

SS 

df 

MS 

F 

Trng  Meth  (TM) 

103.90 

2 

51.95 

1.35 

Aptitude  (A) 

967.39 

2 

483.70 

12.63* 

TM  X A 

37.44 

4 

9.36 

.24 

error  within 

5937.74 

155 

38.31 

Note — 

*p  <.01 
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APPENDIX  E 


Cell  Means  for  Interactions 


Navy:  Number  of  Errors  on  AFAM  Final  Examination 


Aptitude 


High 


Week 


Training  Method 


CM  I 

IMI 

Cl 

1 

6.0 

6.1 

14.7 

2 

5.0 

5.2 

6.2 

3 

7.0 

10.8 

10.8 

4 

3.6 

8.4 

6.0 

5 

2.7 

18.5 

16.4 

6 

6.2 

7.0 

14.8 

7 

6.0 

5.2 

19.8 

Avg 


5.2 


8.7 


12.7 


Medium 


1 

8.2 

10.3 

20.3 

2 

6.1 

8.4 

20.3 

3 

9.7 

10.8 

16.1 

4 

6.1 

7.7 

14.0 

5 

7.3 

6.0 

29.  5 

6 

8.0 

11.9 

45.7 

7 

8.4 

10.9 

19.3 

Avg 


7.7 


9.4 


25.1 


Low 


1 

12.8 

— T275 — 

22.8 

2 

9.0 

14.1 

30.3 

3 

8.2 

14.0 

39.5 

4 

11.3 

15.5 

22.7 

5 

15.2 

8.2 

30.0 

6 

9.8 

19.2 

32.6 

7 

18.3 

9.7 

35.4 

Avg 

12.1 

14.9 

30.5 
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Navy:  AFAM  Time  (hrs) 


Training  Method 

Week 

1 

2 

3 

4 

5 

6 

7 

CMI 

10.1 

11.0 

10.1 

11.7 

10.7 

9.1 

10.9 

IMI 

12.7 

9.9 

11.4 

9.4 

9.3 

9.9 

9.8 

Cl 

32.4 

34.6 

35.0 

34.5 

28.3 

34.9 

35.1 

Marine:  AFAM 

Time  (hrs) 

Week 

Training  Method 

1 

2 

3 

4 

5 

6 

7 

CMI 

13.1 

11.2 

10.8 

14.6 

10.7 

9.3 

12.6 

IMI 

9.1 

11.  8 

9.4 

10.4 

13.1 

8.6 

9.6 

Cl 

35.0 

37.5 

34.9 

35.0 

28.0 

34.5 

36.8 
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Navy:  AMFU  Time  (hrs) 


Aptitude 

Week 

1 

2 

3 

4 

5 

6 

7 

High 

46.0 

41.3 

40.3 

41.7 

49.3 

43.0 

39.7 

Medium 

46.7 

49.0 

48.0 

42.3 

53.0 

46.7 

47.0 

Low 

65.0 

56.3 

70.3 

52.3 

55.7 

56.7 

66.0 

Marine:  Shop  Grade 

Week 

Training  Method 

1 

2 

3 

4 

5 

6 

7 

CMI 

90.0 

90.5 

93.0 

87.0 

91.0 

89.0 

88.0 

IMI 

88.5 

86.5 

90.0 

90.0 

90.5 

87.5 

85.0 

Cl 

92.0 

95.0 

91.5 

95.5 

93.0 

94.0 

94.5 
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APPENDIX  F 


Instructor  Attitude  Questionnaire 


Data  on  instructor  attitudes  was  not  collected  during  the 
period  of  the  evaluation.  More  recently,  however,  a brief 
questionnaire  was  administered  to  44  instructors  in  the  Aviation 
Fundamentals  Course,  Class  P (AFUN) , a combination  of  the  old 
AFAM  and  AMFU  courses,  and  to  54  instructors  in  the  Basic 
Electricity  and  Electronics  (BE&E)  phase  of  the  Avionics  Tech- 
nician Course,  Class  A. 

In  general,  attitudes  toward  CMI  were  somewhat  more 
favorable  in  the  BE&E  course  than  in  the  AFUN  course.  On  a 
summary  question  which  asked  the  instructors  to  indicate  their 
feelings  about  working  with  CMI,  IMI,  and  Cl  by  placing  check 
marks  on  lines  running  from  Very  Happy  (4)  through  Neutral  (2) 
to  Very  Unhappy  (0) , the  BE&E  instructors  had  average  responses 
of  2.6  for  all  three  systems,  whereas  the  AFUN  instructors  had 
average  responses  of  2.3  for  both  CMI  and  IMI  and  an  average 
response  of  3.0  for  Cl.  When  some  of  the  more  general  questions 
from  the  questionnaire  (items  1,  2,  3,  4,  5,  7,  8,  10,  11)  were 
scored  so  that  a score  of  4 would  indicate  a very  favorable 
attitude  toward  CMI  and  a score  of  0 would  indicate  a very  un- 
favorable attitude  toward  CMI,  instructors  in  the  BE&E  course 
had  an  average  score  of  2.3,  whereas  instructors  in  the  AFUN 
course  had  an  average  score  of  1.8.  The  scores  did  not  seem 
to  be  strongly  related  to  variations  in  amount  of  previous  ex- 
periences with  Cl  or  in  the  amount  of  time  that  the  instructor 
had  spent  working  in  a CMI  center. 
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SAMPLE  INSTRUCTOR  ATTITUDE  QUESTIONNAIRE 


Name  

Rate  or  GS  level 


The  following  questions  are  designed  to  find  out  something 
about  the  typical  instructor's  reaction  to  CMI . You  are 
asked  to  provide  some  additional  information  on  your  back- 
ground and  experience,  but  your  answers  will  be  kept 
strictly  confidential.  They  will  not  be  used  to  reflect 
on  you  as  an  individual  or  on  a particular  learning  center. 
Please  think  about  each  question  and  answer  it  seriously, 
since  the  results  can  help  the  Navy  in  its  plans  for  CMI. 

Please  seal  your  completed  questionnaire  in  the  attached 
envelope  and  take  it  to  the  course  office  for  subsequent 
delivery  to  S-39. 

For  each  question,  circle  the  letter  of  the  alternative 
that  best  represents  your  opinion. 


In  what  school  do  you  work? 

a.  AFUN 

b . BE  & E 


How  long  have  you  worked  in  a CMI  learning  center  (or  lab)? 

a.  less  than  2 months 

b.  2-4  months 

c.  4-6  months 

d.  6-8  months 

e.  more  than  8 months 


Have  you  remained  in  the  same  job  for  this  entire  period, 
or  do  you  rotate  from  job  to  job? 

a.  remain  in  same  job 

b.  rotate  from  job  to  job 


If  you  have  remained  in  the  same  job,  provide  a brief 
description  of  the  job  (standard  title  if  there  is  one) . 
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Did  you  ever  work  in  an  IMI  (individualized  but  not  computer 
supported)  learning  center? 

a.  Yes 

b.  No 


Did  you  ever  teach  in  a conventional  classroom,  and  if  so, 
for  how  long? 

a.  never  taught  in  a conventional  classroom 

b.  0-1  year 

c.  1-2  years 

d.  2-4  years 

e.  more  than  4 years 

% Responses 
AFUN  BE  & E 

1.  On  the  average,  students  learn  as  well  in 
a CMI  learning  center  as  they  do  in  a 
conventional  classroom. 


1. 

Strongly  agree 

11 

15 

2. 

Agree 

34 

50 

3. 

Undecided 

20 

13 

4. 

Disagree 

25 

20 

5. 

Strongly  disagree 

9 

3 

2.  Most  students  would  prefer  a CMI  learning 
center  to  a conventional  classroom. 


1. 

Strongly  agree 

11 

6 

2. 

Agree 

43 

26 

3. 

Undecided 

25 

24 

4. 

Disagree 

20 

39 

5. 

Strongly  disagree 

0 

6 

3.  Students  in  a CMI  learning  center  are  less 
motivated  than  students  in  a conventional 
classroom. 


1. 

Strongly  agree 

18 

4 

2. 

Agree 

30 

33 

3. 

Undecided 

11 

20 

4. 

Disagree 

32 

37 

5. 

Strongly  disagree 

7 

6 
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% Responses 


AFUN 

BE  £ 

4. 

Students  in  a CMI  learning  center  suffer 

because  there  is  less  personal  contact 

with  the  instructor. 

1. 

Strongly  agree 

23 

4 

2. 

Agree 

32 

17 

3. 

Undecided 

9 

13 

4. 

Disagree 

30 

52 

5. 

Strongly  disagree 

4 

3 

5. 

The 

Navy  should  increase  its  use  of  CMI. 

1. 

Strongly  agree 

11 

17 

2. 

Agree 

30 

33 

3. 

Undecided 

27 

35 

4. 

Disagree 

20 

11 

5. 

Strongly  disagree 

11 

4 

6. 

I would  be  willing  to  serve  a tour  of  duty 

(or 

take  a job)  teaching  in  a conventional 

classroom. 

1. 

Strongly  agree 

27 

30 

2. 

Agree 

59 

50 

3. 

Undecided 

9 

6 

4 . 

Disagree 

0 

6 

5. 

Strongly  disagree 

4 

9 

7. 

I would  be  willing  to  serve  another  tour  of 

duty  (or  take  another  job)  in  a CMI  learning 

center. 

1. 

Strongly  agree 

14 

28 

2. 

Agree 

47 

50 

3. 

Undecided 

7 

7 

4 . 

Disagree 

11 

7 

5. 

Strongly  disagree 

20 

7 

8. 

I would  prefer  to  work  in  a conventional 

classroom  rather  than  a CMI  learning  center. 

1. 

Strongly  agree 

25 

6 

2. 

Agree 

27 

24 

3. 

Undecided 

25 

44 

4. 

Disagree 

11 

20 

5. 

Strongly  disagree 

11 

6 
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% Responses 
AFUN  BE  & E 


9.  The  instructor's  work  in  a conventional 

classroom  is  harder  than  his  work  in  a CMI 
learning  center. 


1.  Strongly  agree 

14 

11 

2.  Agree 

18 

13 

3.  Undecided 

18 

30 

4 . Disagree 

34 

37 

5.  Strongly  disagree 

14 

6 

Work  as  an  instructor  in  a CMI 

learning 

center  is  more  boring  than  work 
conventional  classroom. 

in  a 

1.  Strongly  agree 

47 

9 

2.  Agree 

34 

22 

3.  Undecided 

2 

13 

4.  Disagree 

9 

44 

5.  Strongly  disagree 

7 

9 

Much  of  my  technical  knowledge 
a CMI  learning  center. 

is  wasted  in 

1.  Strongly  agree 

25 

2 

2.  Agree 

34 

13 

3.  Undecided 

7 

9 

4.  Disagree 

32 

54 

5.  Strongly  disagree 

2 

22 

12.  Instructors  should  rotate  from  one  job  to 
another  within  a CMI  learning  center. 


1. 

Strongly  agree 

41 

30 

2. 

Agree 

45 

57 

3. 

Undecided 

7 

4 

4. 

Disagree 

4 

6 

5. 

Strongly  disagree 

2 

2 
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% Responses 
AFUN  BE  & E 

13.  There  should  be  two  kinds  of  people  in  a 
CMI  learning  center,  one  kind  who  would 
handle  the  relatively  routine  jobs  and 
another  kind  who  would  help  the  students 
with  their  technical  problems. 


1.  Strongly  agree 

7 

7 

2 . Agree 

23 

24 

3.  Undecided 

14 

7 

4.  Disagree 

20 

37 

5.  Strongly  disagree 

32 

24 

I would  enjoy  preparing  instructional 
materials  for  use  in  CMI. 

1.  Strongly  agree 

9 

9 

2.  Agree 

11 

31 

3.  Undecided 

23 

33 

4.  Disagree 

25 

17 

5.  Strongly  disagree 

32 

9 
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